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Outline 

M ultiprotocol Label Switching (M PLS) extends the capabilities of IP routers and ATM Switches in several key ways: 

* It fully integrates IP Routing control with ATM switches, providing native support of IP services such as IP Class of Service and 
IP Multicast on ATM switches as well as routers. 

* It provides support for scalable and flexible IP Virtual Private N etwork (VPN ) services on both switches and routers. 

* It provides support for IP Traffic Engineering, which is the fine-grained adjustment of IP traffic flows according to the resources 
in a network. 


This version of Designing ATM MPLS Networks concentrates on the fundamentals of ATM M PLS network design. These 
fundamentals apply to all ATM M PLS networks, irrespective of the services being offered on them: 

* Choosing M PLS equipment 

* IP+ATM concepts and equipment 

* Designing and dimensioning ATM MPLS points of presence 

* Dimensioning M PLS network links 

¢ IP routing in ATM M PLS networks 

* Dimensioning M PLS Label VC requirements 

* Ongoing network design 

¢ Migrating ATM networksto M PLS 


Open Issues 


Specific Comments & Errata 

¢ 3.1: In Figure 4c it's not clear that the data flow is CPE -- M GX -- eLSR -- MPLS cloud. A revised version of the diagram would 
make this clearer. 

* 3.4: The equations are either correct or “safe” over-estimates. H owever some slightly tighter limits apply to the edge LSRs in 
networks with switches which use unpaired cross-connects, namely the LS1010, 6400 and 8540 M SR. Theseshould be mentioned. 
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¢ 3.x: Add note on this: 
“Actually, there is one thing - if the 7200 is acting as an LSC, then the maximum throughput is limited to full-duplex O C3, as that 
is currently the fastest TSC-BPX link. Thereis no point loading the box up with six OC3 interfaces, unless you actually want five 
of them to run at 20% utilization. Unless you're really loading up the box with _heavy_ edge functions (ag. CAR +M PLS VPN 
Edge +WRED) in addition to LSC function, there won't be much point having an N PE 300 in it.” 

¢ 4.1: Either hereor in an earlier section, it might be useful to illustrate the different M PLS label encapsulations (e.g. shim, VPI/VCl). 

¢ 4.5: Add a diagram showing network with ordinary M PLS links plus IM A links 

¢ 4.5: Add a diagram showing where BN | cards may be used in a network 


Desirable Minor Changes and Additions 
* Add more details on redundancy options and configuration, including dual-controller cooperative redundancy. 
¢ Add more detail on M PLS-over-Frame Relay 
¢ Add more detail on dimensioning the number of edge routers and RPM s, with reference to benchmarking of MPLS Edge +CAR 
+WFQ, etc, is done Include example calculations. 
¢ LDP modes: some discussion on upstream, downstream, conservative, optimistic, etc modes would be useful. 
* 3.x: Expand BPX, M GX issues coverage: 
- LVC allocation 
- “Port” and “trunk” issues, including feeders. (M aterial from recent training slides.) 
- Bandwidth, and what happens when zero bandwidth is assigned to M PLS 
- Dimensioning for LSC, mentioning the 1xO C3 throughput limit 


Desirable Major New Content 
¢ VPNs, with pointer to Packet M agazine article as well as more detail on network-level design issues. For background, refer to the 


VPN design guide & RFC 2457 for more details. 
* MPLS Quality of Service: 
- MPLS CoS configuration 
- MPLS CoS Philosophy & network configuration steps: refer to white paper 
- IP+ATM aspects (service template tradeoffs in BPX ) 
°* IP Traffic Engineering 
* Tools for managing IP VPN networks. 
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1, Introduction 


1.1 MPLS Fundamentals 
The essentials of M ultiprotocol Label Switching (M PLS) operation have been described in other white papers and presentations. 
Some important points will be emphasized here, but for more background see the following: 
* The “IP+ATM Solutions” page at http://wwwin.cisco.com/cmc/cc/cisco/mkt/wan/ipatm/index.htm is a good resource. See 
particularly the “M PLS and IP Quality of Service in Service Provider ATM Networks” white paper under “Internal Information”. 
* The M SSBU SE Toolbox http://wwwin.cisco.com/WAN BU/Tech/index.html contains much information. The links under 
Technical Information ->IP+ATM will be helpful http://wwwin.cisco.com/WAN BU/Tech/ipatm/index.html 
* Online M PLS video training is at http://showtime.cisco.com/index.html 
*The“M PLS VPN Configuration and Design Guide” is at http://wwwin.cisco.com/W W Sales/SE/design/uth/q100/index.html 
* The OSPF version 2 specification is http:/Awww.ietf.org/rfc/rfc2328.txt 
* The “IS-IS for Routing in TCP/IP and Dual Environments” specification is http://www.ietf.org/rfc/rfcl195.txt 
¢ IETF documents on M PLS are at http://www.ietf.org/html.charters/mpls-charter.html. The most important documents are: 
- “MPLS Architecture”, draft-ietf-mpls-arch-05.txt 
- “MPLS Label Stack Encodings”, draft-ietf-mpls-label-encaps-04.txt 
- “MPLS usingLDP and ATM VC Switching”, draft-ietf-mpls-atm-02.txt 
- “LDP Specification”, draft-ietf-mpls-Idp-05.txt 
- “MPLS Support of Differentiated Services by ATM LSRs and FrameRelay LSRs,” draft-ietf-mpls-diff-ext-01.txt 
¢ Other IETF documents on Differentiated Services are at http://www.ietf.org/html.charters/diffserv-charter.html 
* The most important IETF documents on the Border Gateway Protocol are: 
- “A Border Gateway Protocol 4 (BGP-4),” http://www.ietf.org/rfc/rfc1771.txt 
- “Multiprotocol Extensions for BGP-4,” http://www.ietf.org/rfc/rfc2283.txt 
A further informational document shows how BGP can be used to support VPNs: 
- “BGP/M PLS VPNs,” RFC 2457, http://www.ietf.org/rfc/rfc2547.txt 
* The following books on routing, M PLS and related topics are very useful: 
- Halabi, B., “Internet Routing Architectures, “Cisco Press, 1997. 
- Metz, C., “IP Switching Protocols and Architectures”, M cGraw-H ill, 1999 
- Rekhter, et al., “Switching in IP N etworks”, M organ Kaufmann, 1998 
¢ Useful magazine articles are: 
- Feldman, et al., “Evolution of M ultiprotocol Label Switching”, IEEE Communications M agazine, Vol. 36, No. 5, May 1998 
- Metz, C., “Ingredients for Better Routing: Read the Label”, IEEE Internet Computing, Sept/O ct. 1998 
* Some archives on M PLS and related technologies are: 
- http://infonet.aist-nara.ac.jp/member/nori-d/mlr/ 
- http://dcn.soongsil.ac.kr/4insuh/home-mpls.html 
(Thanks to Chris M etz for pointing out many of these resources.) 


The rest of this design guide assumes that the reader has knowledge of M PLS fundamentals. 


1.2 Note on Tag Switching and Terminology 

M PLS is a standardized version of Cisco’s original Tag Switching proposal. M PLS and Tag Switching are identical in principle, and 
nearly identical in operation. This document uses M PLS terminology rather than Tag Switching terminology, as shown in Table 1. 
One exception is the term “Tag Distribution Protocol” (TDP). TDP and the M PLS Label Distribution Protocol (LDP) are nearly 
identical in general function, but use different message formats and some different procedures. The term “TDP” will be used in this 
design guide only when it is important to distinguish TDP from LDP. Otherwise, any referenceto “LDP” in this design guide also 
applies to TDP. 
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N otein Table 1 that the term “Label EdgeR outer” is not used. This is because the equivalent term “EdgeLSR” is used, and 
having two different terms meaning the same thing will lead to confusion. “Edge LSR” is technically the more correct term?. 


Some other terms used are as follows: 

“ATM MPLS” isthe form of M PLS that runsin networks with ATM switches that do M PLS switching. M orespecifically, itis the 
form of M PLS where each different label on a link is represented by a different VC. 

“Packet-based M PLS” means the form of M PLS that runs in networks which do not useATM M PLS. M ore specifically, it is the 
form of M PLS where labels are carried as an extra header attached to each packet. Packet-based M PLS is also known as 
“Non-ATM MPLS", “Frame-based M PLS” and “Router-based M PLS”. The term “Frame-based M PLS” is not used in this 
document, asit seems to imply “Frame Relay,” but Packet-based M PLS does not necessarily have anything to do with Frame R elay. 
A “Packet-based LSR” is a device which manipulates whole packets rather than cells. A router running packet-based M PLS isa 
packet-based LSR. An ATM Edge LLSR is also a type of packet-based LSR. 

“Traditional ATM” switches and networks are those which do not useATM M PLS. Traditional ATM networks may support 
Packet-based M PLS traffic within Permanent Virtual Circuits (PVCs). A traditional ATM switch can support ATM M PLS within 
a Permanent Virtual Path (PVP) which acts a ‘Virtual Trunk’. In any case, the traditional ATM switches do not actually perform 
M ultiprotocol Label Switching— they merely support ‘tunnels’ through which M PLS packets are carried. This is discussed further 
in “4. Migration of M PLS into Traditional ATM Networks” on page 40. 


Table 1 Tag Switching terminology equivalents 


Old Tag Switching Term MPLS Term 

Tag Switching PLS; Multiprotocol Label Switching 

Tag Switch/ed Label Switch/ed 

Tag (short for “Tag Switching”) PLS 

Tag (thing applied to a packet) Labe 

Tag Core Router; TCR Label Switch Router; LSR 

Tag Distribution Protocol; TDP Label Distribution Protocol; LDP (but see text) 
Tag Edge Router; TER Edge Label Switch Router; Edge LSR (see text) 
Tag Switch Controller; TSC Label Switch Controller; LSC 

Tag Switch/ing Router; TSR Label Switch Router; LSR 

Tag VC; TVC Label VC; LVC 

ATM-TSR ATM-LSR 

TFIB LFIB 

Tag Switched Path; TSP Label Switched Path; LSP 


1.3 MPLS Network Structure 

A typical structure for M PLS networks in providers, i.e. carriers or ISPs, is shown in Figure 1. An M PLS network consists of Edge 
Label Switch R outers (EdgeLSRs) around a coreof Label Switch R outers (LSRs). Customer sites are connected to the provider 

M PLS network. Figure 1 shows 9 customer sites and 6 Edge LSRs, but more typically there will be several hundred customer sites 
per edge LSR. The Customer Premises Equipment (CPE) runs ordinary IP forwarding. It typically will not run M PLS. If the CPE 
does run M PLS, it will typically use it independently of the provider. It is important to note that the Edge LSRs are part of the 


1. In packet-based M PLS, particularly in Cisco's implementation, there is nothing special about Edge LSR function compared to LSR function. An Edge LSR is just a 
router-based LSR which happens to have some ordinary IP interfaces. This is often truein ATM M PLSnetworkstoo. TheBPX 8650, for example, hasEdgeLSR capability 
because of two things: the Label Switch Controller (LSC) runs IP routing and LDP, and the LSC is based on a 7200 or 7500 router, which can support ordinary ATM 
interfaces. 
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provider network, and are controlled by the provider. The edge LSRs are critical to network operation, and are not intended to be 
CPE under any circumstances. The provider might locate and manage routers on at the customer sites, but these will be running 
ordinary IP and will be outside the M PLS network itself. 


Ordinary IP in customer sites z= 


J ne Label Switch Routers 
a ys 


_— F* a 
a ee 
: ch 7 
a yj a 
4 =Z == 
| aS : 
a 
LZ Ly 
. a 
/ Edge Label Switch Routers 7 
a 
CPE Provider's MPLS Network CPE 


Figure 1 Typical MPLS network structure. Note that there will typically be many more customer sites than shown in this diagram. 


1.4 Applications for MPLS 

M PLS networks as shown in Figure 1 have three main applications. Typically, two or all three of these capabilities would be used 
simultaneously: 

¢ IP+ATM integration. Because ‘label switching’ can bedoneby ATM switches, M PLSisa way of integrating IP services directly on 
to ATM switches. This involves puttingIP routing and LDP softwaredirectly on ATM switches. Because it fully integrates 1P onto 
ATM switches, M PLS allows ATM switches to optimally support IP services such as IP multicast, IP class of service, RSVP and 
Virtual Private N etworks (see below). O ptimal integration of IP+ATM means that M PLS does not have the scalability and 
complexity disadvantages of overlay schemes like M POA, CSI, and IP N avigator. 

IP Virtual Private N etwork (VPN) services. VPNs form the infrastructure for Intranets and Extranets, which are IP networks on 
which corporations will base their whole business structures. A VPN service is a managed Intranet or Extranet service offered by 
a provider to many corporate customers. M PLS, in combination with the Border Gateway Protocol (BGP), allows one provider 
network to support thousands of customers’ VPNs. So, M PLS with BGP offers a very flexible, scalable, and manageable way of 
providing VPN services on both ATM and packet-based equipment. Even on quite small providers’ networks, the flexibility and 
manageability of MPLS+BGP VPN services are a major benefit. 

IP Explicit Routing and Traffic Engineering. An important problem in current IP networks is the lack of ability to finely adjust IP 
traffic flows to make best use of available network bandwidth. Also absent are related capabilities to send selected flows down 
selected paths, e.g. to select protected trunks for particular classes of traffic. M PLS uses Label Switched Paths (LSPs), which area 
type of ‘lightweight VC’ which can be set up on both ATM and packet-based equipment. The IP traffic engineering capability of 
M PLS uses specially set up LSPs to finely adjust IP traffic flows. 


2. An edgeLSR might be Customer Located Equipment, e.g. in a building basement, but only in circumstances where the location is secure and accessible only by the 
provider. 
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This design guide concentrates on M PLS in ATM networks, although some of the content also applies to packet-based networks. 
This version concentrates on fundamentals of M PLS network design which apply to all ATM M PLS networks, including those 
supporting VPNs and traffic engineering. 


1.5 MPLS and Other IP-over-ATM Schemes 
In ATM networks, MPLS allows ATM switches to directly support IP services, giving maximum efficiency compared to other 


approaches. Traditional IP ove ATM connects routers over Permanent Virtual Circuits (PVCs). M ultiprotocol over ATM (M POA) 

and similar proprietary approaches carry IP traffic over Switched Virtual Circuits (SVCs). Traditional IP over ATM, M POA, and 

proprietary approaches all have similar disadvantages: 

* It is difficult to offer some types of IP services on the networks. For example, IP Class of Service cannot be offered natively by 
traditional ATM switches, and must be offered by translation to quite different ATM Forum Quality of Service concepts. 

* Where IP services are offered, they are difficult to administer. Two levels of routing must be administered: IP routing (via O SPF 
or EIGRP or similar) and PNNI or similar routing for ATM . M POA requires additional administration. Service translations, for 
example IP Class of Service to ATM Quality of Service, also require administration. 

* IP services may be quite inefficient over ATM networks. For example, IP M ulticast over ATM networks is difficult to achieve on 
a large scale due to the interaction of multicast routing, multicast group membership processing and ATM VC maintenance. 

* Theremay be scaling limitations and/or dangerous interactions between IP routing (O SPF, etc.) and the ATM network, leading to 
unstable networks. Traditional IP over ATM can lead to storms of IP routing updates and subsequent network meltdown, if more 
than about 30 OSPF routers are connected in a full mesh over PVCs?. M POA is unsafe when connecting routers to each other, 
and is intended only to connect hosts to routers or hosts to hosts‘. 

* IP services require a substantial implementation and management effort. For example, an M POA implementation requires PN NI, 
SVC signalling, ATM ARP, an ATM ARP server, NHRP, and aNHRP server, in addition to AAL5, IP routing (O SPF, etc.) and an 
IPv4 stack. 


MPLS in ATM networks avoids all of these disadvantages. 


1.6 Steps in Designing MPLS Networks 
The process of designing an M PLS network involves the following steps, which are described in the next sections. These will not 


necessarily be carried out in the following order, but they all need to be covered. 
* Choose equipment types 

* Design points of presence 

* Design the backbone 

* Design the IP routing 

* Check M PLS-specific dimensioning issues 


3. 1f N number of routers are running OSPF and are connected in a full mesh over ATM PVCs, asingle physical ATM link failure may result in ATM -layer re-routing of 
alarge number of PVCs. If this takes too long, or if the ATM network can’t re-route PVCs at all, this results in the effective failure of a large number of PVCs. Thenumber 
of PVCs involved may be of the same order magnitude as N, and even N* in some cases. In any case, it is likely to be seen by O(N) routers, where “O(N)” means “a 
number proportional to N”. So, asingle ATM link failure will cause each of O(N) routers to send a link state advertisement (LSA) of size (at least) O(N) to (N-1) 
neighbors. So, a single event in the ATM network results in O (N 3) to O(N 4) traffic. When a router receives an LSA, it must immediately re-calculate its routing table, as 
it must not forward packets based on old routing information. The processor load caused by a such a “storm” of routing updates may cause the routers to drop or not 
send keepalive packets, which appears to the neighboring routers as further link failures. These lead to further LSAs being sent, which perpetuates the problem. The net 
result is that a full mesh network can go persistently unstable after a single network event. 

This is a fundamental problem, caused by the fact that the routers do not see the state of the ATM links and switches directly. 1S-IS has somewhat better performance 
than O SPF in full mesh conditions because!S-IS has more sophisticated flooding capabilities (these capabilities, specifically the ability to pace flooding and block flooding 
on some interfaces, are also becoming available on O SPF). H owever this does not address the underlying problem. T he solution to this problem is to let IP routing directly 
see the state of ATM links, which is what is done by ATM MPLS. 


4. Thisis not a simple issue to solve. It isa fundamental issue arising from conflict between routing protocols: PN NI routing at the ATM layer can make decisions which 
conflict with O SPF or similar routing at the IP layer. These conflicting decisions can lead to persistent loops. (See the NH RP Protocol Applicability Statement, RFC2333, 
for more discussion on this.) There has been further investigation on router-to-router NHRP at thelETF, but this revealed that router-to-router NH RP wasnot practical. 
The only reliable solution to this problem is to use the same routing protocol at the IP layer and ATM layer. This is exactly what M PLS doesin ATM networks. 
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Class-of-service, MPLS VPN, traffic engineering and other IP services will introduce additional design steps. M any of these are 
outlined in the “M PLS and IP Quality of Service in Service Provider ATM Networks” white paper. (See 
http://wwwin.cisco.com/cmc/cc/cisco/mkt/wan/ipatm/index.htm ) See also the “M PLS VPN Design & Configuration guide” 
http://w wwin.cisco.com/W W Sales/SE/design/uth/q100/index.html 
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2. Choosing MPLS Equipment 


2.1 Structures for MPLS Networks 
M PLS networks have Edge label switch router (LSR) and coreLSR functions as shown in Figure 1. H owever these functions can be 


implemented on various types of equipment, and combined with access equipment there are various ways. These are shown in 
Figure 2. 


Simple Packet-based M PLS 
The simplest M PLS network structure is shown in Figure 2(a). This structure applies to router-only networks, which might use 


M PLS for supporting VPN services or IP Traffic Engineering. In this structure, customer sites are connected directly to router-based 
edge LSRs. The edge LSRs are connected to other LSRs, which are also based on router platforms. The routers are interconnected 


Function: CPE | MPLS | Packet-based MPLS | MPLS | CPE 
Edge Edge 
Device: Edge LSR Label Switch Routers Edge LSR 


a. Simple packet-based M PLS 


Function: CPE | ATM MPLS ATM Label Switching ATM MPLS | CPE 
Device: ATM Edge ATM Label Switch Routers ATM Edge 
LSR LSR 


b.ATM MPLS with router-based Edge LSRs 


Function: CPE | MPLS | Packet-based ATM Label Switching Packet-based |MPLS | CPE 
Edge MPLS MPLS Edge 
Device: Edge LSR ATM Edge ATM Label Switch Routers ATM Edge Edge LSR 
LSR LSR 
c.Mixed ATM and packet-based MPLS 
Function: CPE | Access | ATM MPLS ATM Label Switching | ATM MPLS | Access | CPE 
| Edge Edge 
Device: Access Switch ATM Edge ATM Label Switch Routers ATM Edge Access Switch 


or Concentrator LSR LSR or Concentrator 
d.ATM MPLS with access concentrators 


Figure 2. Devices in MPLS networks 
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by virtually any sort of link: serial, ethernet, packet-over-SO NET, etc., and packets are sent, with M PLS headers, over these links. 
The routers involved will typically be 7200, 7500, or 12000-series Gigabit Switch Routers. M id-range routers (3600 and 4700 
series) might be used in lower-bandwidth applications. 

A variant of this structure is where the point-to-point links between the routers are actually ATM PVCs. These may be used 
during migration to ATM M PLS. This is discussed further in “4. M igration of M PLS into Traditional ATM Networks” on page 40. 


ATM MPLS With Router-based Edge LSRs 
Thesimplest ATM M PLS network structure is shown in Figure 2(b). As with the previous case, customer sites are connected directly 


to router-based edge LSRs, typically 7200 or 7500 series routers. The Edge LSRs are connected by ATM links to the core devices 
which areATM-LSRs. TheATM LSRs may be BPX 8650 IP+ATM Switches, LS1010, 8500 MSR, and later other ATM switches 
such as the M GX 8800 with PXM -45. 


Function: CPE Access + | ATM Label Switching Access + | CPE 
MPLS Edge MPLS Edge 


Device: IP+ATM Edge ATM Label Switch Routers IP+ATM Edge 
Switch Switch 


e.ATM MPLS with integrated ATM edge device 


Function: CPE Access | Backhaul| ATM MPLS | Tunnelling] ATM Label [Tunnelling] ATM Label |ATM MPLS | Access | cre 


1. Edge Switching Switching Edge 


Device: Access Switch ATM ATM Edge ATM ATM LSR ATM ATM LSR- ATM Edge Access Switch 
orConcentrator Switch LSR Switch Switch LSR or Concentrator 


f, ATM MPLS with backhaul & tunnelling 


MPLS (ATM or packet-based) 


Function: CPE Access + 


Access + | 
MPLS Edge 


MPLS Edge 


Device: IP+ATM Edge Aran Edge 
Switch Va Switch 
‘de Ne 
ATM ATM 
g. Dual backbones Switch Switch 


Figure 2. Devices in MPLS networks (continued) 
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TheATM switches carry packets with ATM M PLS labels; this means that, on each ATM link, there isa different M PLS Label 
VC (LVC) for each label. 


Mixed ATM and Packet-based M PLS 
It is possible to have a network with a mixture of ATM MPLS and packet-based M PLS. A simple example of this is shown in 


Figure 2(c). In a network such as this, some links run packet-based M PLS, and some links run ATM MPLS. The devices which 
interface between packet-based MPLS and ATM M PLS arethe same routers which act as ATM EdgeLSRs: anything from a3600 
up to a12000. 


ATM MPLS With Separate Access Devices 
ATM M PLS networks with router-based Edge LSRs may also use separate access devices, as shown in Figure 2(d). This will occur 


when access is required through a device which does not support M PLS services. There three common situations where this will be 

required: 

* Access is required to both IP services, and ATM PVC services, through an access device which does not support M PLS. The most 
common exampleof this isthe M GX 8220. 

* The access device does not yet have software which supports M PLS. This will occur with theAS 5x00, uBR 7246 and |GX 8400 
until they have M PLS support. 

* Higher densities of low-bandwidth access lines can be supported by way of a separate access device, than simply by using an edge 
LSR. 

Customer traffic is carried through the access deviceto the edge LSR. Between the access device and the edgeLSR, thereisa different 

logical link for each customer. This may bea Frame Relay or ATM PVC, ora PPP link. 


ATM MPLS With Integrated IP+ATM Access Devices 
The previous type of network can besimplified if the access device supports edge LSR function as well as Frame Relay, ATM, or 


other access services. This is shown in Figure 2(e). In the caseof IP+ATM edge switches, a single device gives access to both M PLS 
services, and PVC or SVC services. IP+ATM edge switchesincludetheBPX 8680, M GX 8850, 6400 universal access concentrator, 
and later the |GX 8450. 


ATM MPLS Using Traditional ATM Switches 
M PLS networks can use traditional ATM equipment. T his will typically be done as a migration step in introducing M PLS to an 


existing ATM network. Traditional ATM switches can be used in three ways, as shown in Figure 2(f). 

* Backhauling, when the access device is remote from the Edge LSR. The access device is connected to the Edge LSR by PVCs 
switched through an ATM network. 

* Tunnelling through ATM switches between an EdgeLSR and an ATM LSR. In the case, the edgeLSR does not need to be adjacent 
to an ATM -LSR, but can be connected through an ATM network. 

* Tunelling through ATM switches between ATM LSRs. In this case, the core network uses traditional ATM switches as well as 
ATM switches. 

These uses of traditional ATM equipment have disadvantages, and must be used with care. T hese issues are discussed further in “4. 

M igration of M PLS into Traditional ATM Networks” on page 40. 


Dual Backbones 
Providers may sometimes want to keep an existing ATM infrastructure while building a new M PLS infrastructure (either ATM 


M PLS or packet based M PLS) alongside the old infrastructure. Cisco IP+ATM edge devices support this well, allowing customers 
to access both the M PLS network and services, and the old ATM network, even from a single access link. This is shown in 

Figure 2(g). The IP+ATM access devices can be any of those which can be used in Figure 2(e). The network in Figure 2(g) supports 
the same functions and services as Figure 2(e), but the Figure 2(g) network requires more equipment. 
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2.2 Choosing Cisco MPLS Equipment 


Choosing ATM MPLS Edge Equipment 

There are four main considerations when choosing ATM M PLS edge equipment: 

* Type of services to be offered: IP+ATM, i.e. end-to-end PVC and SVC services as well as IP services, or just IP 

* Type of access lines 

¢ Number of access lines 

* Requirements for redundancy and reliability. K ey issues are whether the equipment can minimise (warm standby) or completely 
prevent (hot standby) disruption to data flow in the case of software or hardware failure, and whether individual components, 
eg. port cards, can be hot-swapped. H ot standby means zero or almost zero (under 1 second) interruption to end-to-end data 
flows in the case of equipment failure, with no re-routing beyond the failed equipment. 

Redundancy levels for M PLS edge devices can be classified as follows: 

* None: The edge device has no redundancy features. The network must rely on re-routing for reliability. Customer sites must be 
dual-homed to two different access devices to ensure reliable service. 

* Processor redundancy: The edge device has a redundant pair of processors and backplanes with warm or hot standby. Since port 
cards are not in redundant pairs, the network must still rely on re-routing for reliability. Customer sites must still be dual-homed 
to ensure reliable service in case of port card failure, but they may be dual-homed to two different cards on the same access device. 

* Full redundancy. The edge device has redundant processors and port cards. Customer sites will receive reliable service from the 
edge equipment even if they are single-homed. 

Equipment recommendations based on these requirements areshown in Table 2. In future, it is likely that Cisco’s will support M PLS 

on most or all mid-range and high-end routers, most or all ATM switches, and most or all access products with routing capability. 

Consequently, more M PLS edge devices will become available in the future. Details of all Cisco products are available from 

http://www.cisco.com 

There are some practical issues to bear in mind when examining the product specifications for products to be used as ATM Edge 

LSRs: 

* When considering the number of access lines supported, take into account the card slots used by the ATM M PLS interfaces. 
For example, a Cisco 7206 router http://www.cisco.com/warp/public/cc/cisco/mkt/core/7200/prodlit/c7200_ds.htm has 6 card 
slots, and can nominally support 48 ethernet ports (or 8 per slot). H owever, when using the Cisco 7206 router as an ATM Edge 
LSR, at least one slot must be used for an ATM interface. So, the actual ethernet port capacity of a Cisco 7206 ATM EdgeLSR is 
40 ethernet ports. 

* Some edge LSRs use card slots for cards performing the Edge LSR function, and these need to be taken into account. 

For example, the Cisco 6400 Universal Access C oncentrator 
http://www.cisco.com/warp/public/cc/cisco/mkt/access/dslaggr/prodlit/6400_ds.htm has 8 card slots, but when it acts as an Edge 
LSR, at least one of these slots must be used for a N ode Route Processor card, and not a line card. 

* Some edgeLSRs can deal with a throughput of M PLS traffic which varies with the number of processing cards. 

For example, at the time of writing the current N ode R oute Processor (N RP) card in a Cisco 6400 Universal Access Concentrator 
can handle roughly 150M b/s full-duplex of M PLS edge traffic. This means that, assuming a typical activity fraction of 25%, a 
Cisco 6400 with oneN RP can handle four 0 C3/STM -1 access lines; whereas a Cisco 6400 with two NRPs can handle eight 
OC3/STM -1 lines. 

The number of slots taken by processor cards also must be taken into account when calculating the number of access lines 
supported for any particular configuration. 


Choosing ATM Label Switch Routers 

There are five main considerations when choosing ATM LSRs: 
* Type of trunks 

¢ Number of trunks. 


Designing ATM MPLS Networks v0.51 
Cisco Internal Use Only. 
Copyright © 1999 Cisco Systems, Inc. All Rights Reserved. 
Page 12 of 56 


Table 2 Choosing MPLS Edge Equipment for ATM M PLS networks 


Equipment Type ofServices | Access Lines® Redundancy Support | Comments 

3600 router IP only Relatively small numbers of async, modem, None Small number of LVCs supported on ATM 
serial/frame relay, 10mb/s ethernet, ISDN cards will lead to limitations on M PLS 
BRI & PRI, HSSI, E1/T1 serial, fast ethernet, network size. Not recommended for 
0C3/STM-1ATM, voice interfaces, and provider ATM MPLS networks. 
others. 

4700 router P only Relatively small numbers of serial/frame None Small number of LVCs supported on ATM 
relay, 10M b/s ethernet, ISDN BRI, E1/T1 cards will lead to limitations on M PLS 
serial, fast ethernet, E3, T3 or OC3/STM-1 network size. Not recommended for 
ATM, and others. provider ATM MPLS networks. 

7200 router P only Serial/frame relay up to E1/T1, 10M b/s & None Minimum recommended for provider 
fast ethernet, ISDN BRI, HSSI, high-speed networks. 
serial, E3, T3 or OC3/STM-1ATM, PA-A2 CES-ATM port adaptors do not 
packet-over-SONET/SDH and others. currently suppor MPLS. me 

7505 router, or 7507, P only Serial/frame relay or ISDN up to E1/T1, Warm-Standby Processor 

7513, or 7576. 10M b/s, fast, & gigabit ethernet, HSSI, Redundancy with dual 
high-speed serial, ATM, RSPs. 
packet-over-SONET/SDH and others. 

12008/12012 IP only POSIP and ATM atOc3 to OC48 rates, and Warm-Standby Processor Suitable for high-speed peerings between 
gigabit ethernet. Redundancy providers. 

Catalyst 5500 with IP+ATM 10M b/s & fast ethernet, E3, T3, OC3/STM -1 None The Cat 5500 is primarily a LAN switch, but 

Route Switch ad OC12/STM -4ATM, and others. also has limited Edge LSR capability. 

M odules The Cat 5500 may only be connected to an 


ATM MPLS network by tunnelling, as shown 
in Figure 2(f). This is discussed further in 
section 4. on page 40 


6400 IP +AT ATM at E3/T3 to STM-4 rates, also ethernet Warm-Standby Processor MPLS support is not yet shipping. 
and fast ethernet. Redundancy 

M GX 8850 IP +AT. High numbers of 56k/64k Frame Relay, T1/E1 Full Warm-to-Hot Standby, 
Frame Relay, channelized, and ATM, and (but with FCS limitations) 


higher-speed Frame Relay, serial and 
channelized T3. 


BPX 8650 IP +AT High numbers of 56k/64k Frame Relay, T1/E1 Excellent redundancy in (See BPX 8680.) 
Frame Relay, channelized, and ATM, ATM at general, but there is a 
E3/T3 to STM -4 rates, and others. single point of failure for 
Edge LSR function. 
BPX 8680 IP +AT High numbers of 56k/64k Frame Relay, T1/E1 Full Warm-to-Hot Standby BXM trunk cards must be used. BXM cards 
ATM, Frame Relay and channelized. Also (but with FCS limitations) are required. MPLS is not supported on BNI 
ATM at E3/T3 to STM -4 rates, and others", cards, except ifthe BNI cards are used as 


Feeder Trunks. 


BCC cards must be BCC3-64 or later. BCC4 
cards are strongly recommended. 


a. For more details see http://www.cisco.com. 
b. Note that the highest ATM bandwidth density supported by the 12000 series port cards is 1x OC12 per slot. Since all traffic inan ATM edge LSR must go through an ATM interface 
into the ATM MPLS network, this relatively low ATM bandwidth density of the 12000 limits its capacity as an ATM Edge LSR. 

c. AtFCS, the M GX 8800 will have hot-standby1:N redundancy capability for customer access lines, hot standby control for PVCs and hot-standby trunks. However it will not yet support 
redundant Edge LSRs processors (Route Processor Modules), so it will be effectively non-redundant as an edge LSR device. 1:N warm standby redundancy for RPMs is scheduled for 
release in CY2000, 
d. Extra 7200 or 7500 routers (or “Label Switch Controller” packages) may be required to act as Edge LSRs E3/T3 or faster ATM access lines are used. If IP service is to be supported 
for large numbers of ATM links at T3/E3 rates and above, its is more cost-effective to use separate, stand-alone, routers. 

e. .The BPX 8680 can include up to 16 M GX8850 shelves, with redundancy features as described above. Full redundancy for the combined device relies on redundancy for the label 
switch controller for the BPX 8600 shelf. A re dun and ant configuration using two simultaneously active controllers will be supported in BPX software release 9.3. 


¢ Number of connections supported 
¢ Whether VC M erge is required 
* Requirements for redundancy and reliability, as discussed above. 


Equipment recommendations based on these requirements are shown in Table 3. In future, it is Cisco’s intention to support M PLS 
on all ATM switches. Consequently, moreAT M -LSRs will become available in the future. In addition, any traditional ATM switch 
can be used in a Cisco M PLS network if tunnelling is used, subject to significant limitations. This is discussed further in “4. 

M igration of M PLS into Traditional ATM Networks” on page 40. 
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Table 3 Choosing ATM-LSRs 


Number of 
Type and Number of ATM Connections VC Merge 
Equipment Trunks@ Supported Supported? | Redundancy Support | Comments 
MGX 8850 with PXM1 |) 4x0C3/STM1 8k? No Full Warm-to-Hot Standby The M GX 8850 is intended 
primarily to be an edge LSR, but 
will also has limited ATM -LSR 
capability. This capability may not 
be available until late in CY 2000. 
LS1010 32 x T1/E1 with Inverse M ultiplexing 64k Yes None 
over ATM (IMA), 32x T3/E3, 32 x 
OC3/STM -1, 8 x OC12/STM -4 
6400 16 x T3/E3, 16 x OC3/ATM-1, 64k Yes Warm-Standby Processor ATM-LSR support on this switch is 
8x OC12/STM -4 Redundancy not yet shipping. 
BPX 8650 144 x T3/E3, 96 x OC3/STM -1, 192k cy2000° Some redundancy features All MPLS interfaces must be on 
24 x OC12/STM -4 at FCS, full redundancy is BXM cards. 
possible later. BCC cards must be BCC3-64 or 
later. BCC4 cards are strongly 
recommended. 
8540 MSR 64x T1/E1 with IMA, 64 x T3/E3, 256k Yes Warm-Standby Processor Some trunk cards are not yet 
64 x OC3/STM -1, 32 x OC12/STM -4, Redundancy shipping. 
8 x OC48/STM -16 
M GX 8800 with 192 x T3/E3, 144 x OC3/STM-1, 384k Yes Full Warm-to-Hot Standby PXM -45 cards are not yet shipping 
PXM-45 card(s) 48 x OC12/STM -4, 12 x OC48/STM -16 
a. lf redundant pairs of trunks are used, the number of trunks supported will by half the numbers shown. 
b. 16k full-duplex connection legs are supported on the PXM card of the M GX 8850. If both legs of all connections are on the PXM card, then 8k connections are supported. 
c. VC Merge will be supported in BPX software release 9.3, with Enhanced BXM cards. 
d. The B PX8650 has supports hot standby trunks and switching fabrics. Full redundancy relies on redundancy for the label switch controller for the BPX 8600 shelf. A redundant 


configuration using two simultaneously active controllers will be supported in BPX software release 9.3. 


Label Switch Routers Not Based On ATM Switches 
LSRs other than ATM switches may be also be used. In particular, the following routers may be used asLSRs: 


* 3600 & 4700 series routers (low-bandwidth applications only) 

¢ 7200 & 7500 series routers 

* 12000 series Gigabit Switch R outers 

Using these routers, M PLS may be supported over virtually any link type: ATM , Packet-O ver-SON ET, Ethernet, etc. R outer-based 
LSRsdo not support nativeATM Virtual Circuit connections, and all except the 12000 series have relatively low capacity compared 
to Cisco ATM -LSRs. 


2.3 IP+ATM 
IP+ATM capability is a major strength of Cisco ATM switches, and is one of the most important selling advantages of them when 


competing against other vendors. “IP+ATM” is Cisco’s name for equipment which simultaneously supports traditional ATM 
services (PVCs, SVCs, SPVCs, PVPs, etc.), as well as optimized IP transport using M PLS. Theconcept of anIP+ATM switchisshown 
in Figure 3(a). A single switch contains two logically separate switches: an M PLSATM Label Switch R outer (LSR) optimized for IP 
transport, and a traditional ATM PVC/SVC switch. Each trunk can support both PVCs (or SVCs, etc.), and M PLS Label VCs 
(LVCs). 

Although an IP+ATM switch contains logically separate switches, it physically contains only one switch. H owever it contains 
two (or more) separate sets of control software. One set of software controls PVCs, SVCs, etc., and the other control software 
controls M PLS. These controllers act independently, allowing the single physical switch to act as two (or more) virtual switches. In 
switches such as the BPX 8650 and M GX 8850, this independent control is implemented using a switch control interface (SCI). 
(The switch control interface used is the Virtual Switch Interface (VSI).) The SCI allows two (or more) separate controllers to 
independently control a single switch, as shown in Figure 3(b). The M PLS control software is physically located in a Label Switch 
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Figure 3 IP+ATM 
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Controller (LSC). In the BPX 8650, the LSC is a device separate from the main switch shelf. In the MGX 8850, theLSC is based 
ona Route Processor M odule (RPM ) in the switch shelf itself. O ther SCI controllers may besoftwarerunning on the switch control 
card°. The LS1010 and 8540 MSR implement similar functionality to the SCI using internal software interfaces. 

In order that the control planes can act independently, the SCI “slave” process(es) in the switch must allocate resources two the 
different “control planes” (M PLS, PNNI, etc.). In the BPX 8650, resources for AutoR oute PV Cs are reserved in a similar way. The 
resources partitioned in this way are: 
¢ VPI/VCI space on trunks. Each control plane gets a range of V Pls to use. 

* Bandwidth.Each control planeis guaranteed a certain bandwidth for Connection Admission Control (CAC) purposes. With “soft 
partitioning,” there can bea pool of bandwidth which is shared between control planes for CAC purposes. Even with “hard” 
partitioning, spare bandwidth unused by a control plane is available on a cell-by-cell basis to other control planes. 

* Traffic queues. One of the keys to IP+ATM is that M PLS traffic gets different traffic queues on the switch than the PVC and SVC 
traffic. This means that M PLS traffic can be handled by queues which directly support the M PLS “Class of Service” concept. The 
alternative is manually-configured translations to ATM Forum Service Types. The need for these translations is one of the main 
disadvantages of |P-over-ATM schemes apart from M PLS, and IP+ATM avoids this disadvantage. 

Part of the configuration process for IP+ATM switches is the assignment of these resources to the different control planes. this 

involves creating different “partitions” of link resources for the different control planes, as shown in Figure 3(c). 


Use of IP+ATM 

IP+ATM can be used to offer M PLS services, and PVC, SVC, etc. services, on the same network. This means that all (or many) 
switches in the network act as both ATM -LSRs and traditional ATM switches, as in Figure 3(a). The traditional ATM services can 
also be used in conjunction with an M PLS service. Figure 3(d) shows the use of a PVC to connect ordinary IP traffic from customer 
site through to an ATM EdgeLSR.A PVC used in this fashion is called an “M PLS Access PVC”. Other PVCsare“ traditional PVCs” 
as part of a traditional end-to-end PVC service. Thetraffic from the Edge LSR can then befed back through the AT M -LSR function 
in thesame switch that supports the M PLS Access PVC, or alternatively through a different switch. In any case, the end-to-end data 
path for customers’ IP traffic can include both M PLS Access PVCs and M PLS Label VCs. 

Anintegrated IP+ATM Edge Switch, such as theM GX 8850 or Cisco 6400, contains ATM -LSR function, as well as traditional 
access switch and PVC switching function. In addition, the Edge LSR function is also integrated into the device. This shown in 
Figure 3(e). In the M GX 8850, routing function is supported by way of Route Processor M odules (RPM s), and Node Route 
Processor (NRP) modules are used in the Cisco 6400. Each RPM or NRP actsasan EdgeLSR.IntheM GX 8850, oneof theRPM s 
will simultaneously act as an LSC and an EdgeLSR. Theimplications of this can been seen more clearly in Figure 7 on page 26. 


5. In the case of the BPX 8650 and MGX 8850, AutoR oute software, which controls PVCs, runs on the switch control card. PN NI control will be added to the BPX 
8650 as a separate controller. Later MGX 8850 releases will have PN NI software running on the switch control card. 
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3. Designing MPLS Networks 

The goal of designing an M PLS network prior to installation is to produce a network which will operate correctly, and which will 
comeat least reasonably close to meeting performance goals. Because of theinherently connectionless nature of IP traffic, customers 
will not be able to tell a carrier exactly what traffic they want to send where. Because of this, it is not possible to perfectly design a 
network ahead of time. The initial design steps result in a working network: 

* Design Points of Presence 

* Dimension backbone links in the network 

* Design IP routing 

* Dimension M PLS Label VC space 

The final design step is an on-going process of optimizing the network design: 

* Refine the design once the network is operational 

These steps are considered in the following subsections. 


3.1 Points of Presence Structures 
The design of Points of Presence (PoPs) for an ATM M PLS network is constrained by: 


* The choice of access line types and equipment for a network, as discussed previously. 
* Location of PoPs, which is largely determined by where the cities are. 

* The population of user sites surrounding each location. 

Some typical PoP designs are shown in Figure 4. 


Single ATM Edge LSR 

WhereasingleEdgeLSR deviceis sufficient for supporting thenumber and types of access lines in a Point of Presence (PoP) location, 
then the simple structure shown in Figure 4(a) is sufficient. N umerous access lines (typically tens or hundreds) are brought into a 
single Edge LSR, which is connected to the rest of the ATM M PLS network. Numbers and types of access lines supported by single 
EdgeLSRs are described in Table 2. 


Multiple Edge LSRs and an ATM-LSR 

A PoP may requiremorethan oneedgeLSR because of a largenumber of access lines to be supported at that location. Alternatively, 

different types of edgeLSR might be required because of different types of access lines to be supported. Where there are several edge 

LSRsin a PoP, it makes sense to also include an ATM -LSR. This is shown in Figure 4(b). The ATM -LSR: 

* Locally switches traffic going between different edge LSRs in the PoP 

* Concentrates traffic going from the PoP onto asingle set of ATM M PLS links. The alternative would be either separate sets of 
links to all edge LSRs, or using one edge LSR to carry traffic to the others. 

* Improves scalability of routing. O nly one set of IP routing protocol (e.g. OSPF) peerings are required from the ATM -LSR to other 
points in the M PLS network. Without the ATM LSR, separate peerings would be required from all edge LSRs. 

Depending on reliability requirements, redundant pairs of links would be used between the edge LSRs and the ATM -LSR. 


Designing ATM MPLS Networks v0.51 
Cisco Internal Use Only. 
Copyright © 1999 Cisco Systems, Inc. All Rights Reserved. 
Page 17 of 56 


ATM MPLS links 
to ATM-LSRs or 


a. Single Edge LSR other Edge LSRs 


ATM MPLS links 
to ATM-LSRs or 
other Edge LSRs 


BPX 8650 


MGX c. Edge LSR PoP with BPX 8650 and 
8220 M GX 8220 access concentrators 


Physical view 


Functional view 
Route Processor M odules or 


Node Route Processors d. 6400 or MGX 8850 Edge LSR 


ATM or other 


e. Stand-alone ATM-LSR f, Stand-alone router-based LSR 


Figure 4 Point of Presence Structures for ATM MPLS networks 


Designing ATM MPLS Networks v0.51 
Cisco Internal Use Only. 
Copyright © 1999 Cisco Systems, Inc. All Rights Reserved. 
Page 18 of 56 


Edge LSR PoP With BPX 8650 and M GX 8220 Access Concentrators 
An extension to the previous model is to use traditional access concentrators in addition to edge LSRs and an ATM LSRs. 


Circumstances where this is appropriate are discussed in “2.1 Structures for M PLS N etworks” on page 9. One example of this type 

of PoP uses M GX 8220 access shelves, 7200 or 7500 edgeLSRs, and BPX 8650. Thisis shown in Figure 4(c). IP traffic from access 

concentrators is carried in ATM PVCsto a edgeLSRs. These may be carried through the same BPX 8650 that actsasan ATM -LSR, 
as this is an |P+ATM switch. EdgeLSRs in such a PoP may have three different types of configuration: 

* Typically, oneor moreedge LSR swill be dedicated to dealing with IP traffic on the edgePVCs. The router labelled ‘x’ in Figure 4(c) 
has this configuration. It must have at least two ATM interfaces: one for access PVCs (from the access concentrator) and one for 
ATM M PLS traffic. 

* An edgeLSR dealing with access PVCs may also have customer access lines directly terminating on it. Router ‘z’ in Figure 4(c) 
does this. 

* There may also be edge LSRs in the PoP which don’t deal with access PV Cs at all, and have only directly-connected access lines. 
Router ‘y’ has this configuration. 

N ote that the Label Switch Controller (LSC) in the BPX 8650 can act as an EdgeLSR simultaneously with performing its LSC 

function. H owever, useof an LSC as an EdgeLSR is not recommended for providers who consider the separation of M PLS control 

functions from data forwarding functions to beimportant. Asan edgeLSR, an LSC can perform any of the three functions discussed 
above. The number of Edge LSRs required in the PoP depends on: 

* The total number of access lines 

* The total bandwidth of the access lines, calculated from the average utilization. For example, the sum of the access lines 
bandwidths might be (say) 1Gb/s, the utilization might not exceed 500M b/s. 

The capacity of a 7200 or 7500 router running M PLS edge function is roughly the same as its ordinary IP capacity using Cisco 

Enhanced Forwarding (CEF). For example, a 7200 router with an N PE 200 processor can support close to 200 M b/s of M PLS edge 

traffic, at normal IP packet sizes®. It can support M PLS edge function for about 700 access lines. [These figures are preliminary and 

need confirmation. ] 


Cisco 6400 and M GX 8850 Edge LSRs 
TheM GX 8850 and Cisco 6400 integrate the functions described in the previous example into a single device, illustrated in 


Figure 4(d). It consists of: 

* A multiservice access concentrator with various types of Frame Relay and ATM access lines, as well as Circuit Emulation lines. 
Voice access capability and other types of access lines will be added later. 

* Oneor more edge LSRs. Each edge LSR isa Route Processor M odule(RPM ) card in the case of the M GX 8850, or aNodeRoute 
processor (N RP) card in the case of the 6400. The number of RPMs or NRPs required to act as Edge LSRs depends on: 


- The total number of access lines 

- The total bandwidth of the access lines, downrated according to the utilization. For example, the sum of the access lines 
bandwidths might be (say) 1Gb/s, the utilization might not exceed 500M b/s. 

An RPM with an NPE150 processor can support M PLS edge function for 700 access lines. It will support close to 150M b/s of 

M PLS edge traffic, at normal IP packet sizes’. The NRP is similar. These limits will be raised in future due to software and 

hardware improvements. 


6. If additional edge functions such as CAR and WRED areused, then performance may be affected. In these circumstances, the performance of the routers needs to be 
verified for the particular combination of features which will be used. N ote also that voice-over-IP packets are very short, and will have significantly lower throughput 
than indicated here. 


7. Again, the performance needs to be verified with the particular combination of edge functions to be used in addition to M PLS edge function. 
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* ANATM-LSR.IntheM GX 8850, oneof theRPM cards acts as a Label Switch Controller. It may perform both LSC function and 
edge LSR function simultaneously, if desired. Use of an RPM for simultaneous LSC and EdgeLSR is not recommended for 
providers who consider the separation of M PLS control functions from data forwarding functions to be important. In the Cisco 
6400, the main N ode Switch Processor also acts an LSC. 

The 6400 and M GX 8850 also havelIP+ATM capability. N ote that these functions are being phased in over time. In particular, LSC 

function will not be available until later. This has implications to network design which are discussed in “4.5 Examples of H ybrid 

ATM Network Equipment” on page 52. 


Stand-Alone ATM-LSR s 

Some sites in a network may have purely a switching role. In an ATM M PLS network, these sites will consist of a single ATM -LSR, 
as shown in Figure 4(e), or possibly a redundant pair of ATM -LSRs. The ATM -LSR would typically bea BPX 8650, 8540 M SR, or 
later an M GX 8800 with PX M -45. In somenetworks, it may make senseto usea router-based LSR instead, as shown in Figure 4(f). 
A 7500 or 12000-series router may be suitable for this application. Core LSRs usually have Edge LSR capability as well. For 
example, the BPX 8650 has limited edge LSR capability as part of its Label Switch Controller. 


3.2 Dimensioning an MPLS Network’s Links 

This section is intended to be a guideline to the steps of dimensioning the links in a small M PLS network, but it is not the only way 
of doing this. Different providers will have their own unique procedures for designing and running networks, but they all will be 
roughly similar to this. Recall that it is not the purpose of initial design to produce a perfect network. Several approximations are 
made in this process which lead to a network which works, but is not necessarily optimal. The last step in the design process is to 
optimize the network, and this is discussed in “3.5 On-going N ework Design” on page 38. 


1. Design edge points of presence and their layout. 
The first step in M PLS network design will to be to choose the size, type, and layout of the Points of Presence according to the 
considerations described above. An example is shown in Figure 5. This example is based on a network in Australia. Australia 
was chosen for this example because it is small enough to makea suitable realistic example. The edge PoPs shown in Figure 5(a) 
are chosen based on the estimated customer link demand shown in Figure 5(b). BPX 8600-based edge LSR PoPs (which consist 
of several M GX 8800 shelves and a BPX 8650 for aggregation) are used in Sydney and M elbourne, which have the largest link 
bandwidths and number of links in this example An M GX 8800 is used in Brisbane. Adelaide and Perth are smaller centres 
which can, in this example, be adequately served by router-based PoPs. 

2. Estimate traffic from each point of presence 
Based on the total access line bandwidths, an estimate on the total traffic sent from customers into each PoP can be made. A 
‘busy-period’ estimate should be used, e.g. of the rate during the busiest minute of the day. This is to ensure adequate 
dimensioning. A conservative estimate would be the total of the access line bandwidths at the PoP, as in Figure 5(b). However 
it will often be reasonable to take a somewhat lower estimate, e.g. 50% of the total access bandwidth, as shown in Figure 5(c) 

3. Estimate the traffic matrix 
The exact process for this step will vary from network to network. In Australia, for example, the two main business centres are 
Sydney and M elbourne, with Sydney being slightly larger. In a largeM PLS network for interstate business IP traffic, a reasonable 
first approximation may be that 50% of traffic will go to Sydney, 40% to M elbourne, 5% to Brisbane, and 2.5% to Adelaide 
and Perth respectively. An existing service provider would probably already have estimates for traffic patterns for their region. 
Based on the estimated traffic distribution percentages, and the total PoP traffic from Step 2, a traffic matrix can be estimated. 
Thetraffic matrix for this example is in Table 4. In a typical network, this matrix will be very roughly symmetrical. For example, 
in Table 4, the traffic from Sydney to Adelaide is 12.5M b/s, but thetraffic from Adelaide to Sydney is 25M b/s. If the traffic were 
more asymmetrical than about 2:1 or 3:1, then there may bean error in traffic estimates or modelling. 

4. In IP networks, traffic from x to y will often flows along the same path (but in the reverse direction) as traffic from y to x. 
Although this can be overridden by numerous routing protocol features, it maybe useful to assume that this will happen, 
particularly in small networks. Because of this, it may be easier to use bidirectional traffic flows rather than unidirectional flows 
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Table 4 Network example: uni-directional traffic matrix 


Traffic Source 

Traffic Distribution 

Destination Percentage Adelaide Brisbane Melbourne | Perth Sydney 
Adelaide 2.5% 1.25 2.5 10 1.25 12.5 
Brisbane 5% 25 5 20 25 35 
Melbourne 40% 20 40 160 20 200 
Perth 2.5% 1.25 2.5 10 1.25 12.5 
Sydney 50% 25 50 200 25 250 
Total 100% 50M b/s 100M b/s 400M b/s 50M b/s 500M b/s 


in an initial network design. The estimated bidirectional flows for the example network are shown in Table 5. The bidirectional 
traffic bandwidth between Adelaide and Sydney, for example, is taken to be25M b/s, which isthe maximum of theunidirectional 
bandwidth from Sydney to Adelaide (12.5M b/s) and the bandwidth from Adelaide to Sydney (25M b/s). Forming bidirectional 
flowsin this way will tend to slightly overestimate the traffic in the network. This is useful as a conservative first approximation. 


Table 5 Network example: approximate bidirectional traffic flows 


Adelaide Brisbane Melbourne | Perth Sydney 
Adelaide 125 
Brisbane 2.5 5 
Melbourne 20 40 160 
Perth 1.25 25 20 1.25 
Sydney 25 50 200 25 250 


5. Design the layout of the backbone network 
The layout of the backbone will involve consideration of 
- Geographic layout, i.e. good locations for nodes 
- Network-level redundancy, i.e. having multiple paths to each destination 
- Redundancy of trunks 
The network layout chosen in this example is shown in Figure 5(d). M any layouts are possible. The one chosen consists of a 
combination of a partial ring, linking adjacent nodes at the outside of the network, and a star, connecting nodes back to an extra 
ATM-LSR in the coreof thenetwork. T his provides a good degree of network-level redundancy, with at least two paths between 
each pair of nodes. The extra ATM -LSR node is placed in Bourke, a town which is roughly equidistant from four of the five 
customer PoPs. With a good degree of network-level redundancy, it is not essential to have redundant trunks, because it is 
possible to reroute M PLS Label VCs®. In this example, most trunks are chosen to be non-redundant for economy. A redundant 
pair of trunks is used for the link which is expected to have the heaviest utilization, namely between Sydney and M elbourne. 


8. In traditional connection-oriented networks, re-routing of virtual circuits is a last resort to be used only when all other redundancy mechanisms have failed. This is 

becauseit inevitably involves disruption of customer traffic for many seconds or minutes as all circuits are re-routed. In traditional IP networks, re-routing is a much less 
severe issue, as packet flows can be switched from one link to another almost instantaneously, once the IP routing protocol has converged. M PLS networks lie between 
theses two extremes. Rerouting in M PLS networksis particularly feasible if VC M ergeis used, for two reasons: VC merge reduces the number of VCs which are used 
in the network, and it reduces the scope of changes required in connections when re-routing does occur. 
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Figure 5 Network design example - geographic layout 
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Figure 5 Network design example - geographic layout (continued) 
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6. Estimate link flows based on the estimated traffic 
This involves calculating the paths taken by traffic through the network backbone. With IP routing protocols such as O SPF, this 
is a straightforward procedure, as the IP traffic will follow the minimum-hop path unless administrative costs are used. Where 
there are two or more minimum-hop paths, the traffic will be approximately balanced across them. The process of calculation 
of link flows for the traffic in Table 5 is shown in Figure 6(a), and the totals in Figure 6(b) 

7. Assign link capacities 
Based on the estimated link flows, link capacities can be assigned to the links in the network. This would generally involve 
choosing the next standard link size (T 3/E3, STM -1, etc.) larger than the link flows just calculated. This is illustrated in 
Figure 6(c). 

8. Adjust for redundancy 
Where redundant trunks are not used, and the network relies on re-routing for reliability, it may be necessary to adjust link 
bandwidths to ensure that there is sufficient capacity to deal with link failures. For example, if the link labelled ‘r’ in Figure 6(c) 
fails, then links ‘s’ and ‘t’ will need to carry some or all of its traffic. The load on these links would then exceed E3 rates, so an 
STM -1 link (or multiple E3 links) would be required for each of links ‘s’ and ‘t’. Similarly, if link ‘u’ failed, then link ‘v’ would 
require more than E3 capacity. Also, if link ‘y’ failed then the offered load for ‘w’ would exceed E3. So, the final allocation of 
link bandwidths is as shown in Figure 6(d). 

9. Check whether the selected equipment is adequate 
This involves checking in Table 2 and Table 3 whether the selected PoP equipment can support the number and size of links 
chosen in the network design. The network in this case would pass this check But for example, assume the M elbourne PoP had 
used an M GX 8850 instead of a BPX 8680. This PoP needs two STM -4 links and two STM -1 links, which is not yet supported 
onan M GX 8850. So, in this hypothetical example, the PoP would need to be re-designed, e.g. by using a BPX 8680 instead of 
an M GX 8850. 
N ote that any such re-designs, if required, area relatively minor issue. M any different types of Cisco equipment can be used in 
ATM MPLS PoPs, and it will usually be found that a PoP can be built to meet the requirements of one location, simply by using 
combinations of equipment used at other locations. A BPX 8680, for example, combines several M GX 8850 shelves. 


Note on Redundant Pairs of ATM Links 
There are three main ways of achieving changeover for a redundant pair of ATM links: 


* Data link-level changeover. This is the normal link redundancy mechanism in traditional ATM networks. Changeover occurs 
because of physical and data link-layer monitoring in the ATM switch or ATM -LSR hardware. T heswitch hardware also typically 
sets up a copy of all virtual circuit state on the backup link. In a switch with data link-level redundancy, any singlelink failure will 
typically result in close to zero data loss on any virtual circuits. In addition, the network layers and routing (IP or PNNI, etc.) will 
not be affected by the link failure, or even be aware that it has occurred. H owever with data link redundancy, the backup link is 
not available to carry data except in the case of failure of the main link. Depending on how it is implemented, SON ET Automatic 
protection Switching (APS) may bea form of data link redundancy. On theM GX 8850 and BPX 8650, SON ET APS changeovers 
result in no change to the interfaces as seen by connection routing, and no loss of connection state. T his does not necessarily apply 
to APS on other Cisco equipment. 

Inverse multiplexing over ATM (IMA). IMA carries distributes data over a group of links by distributing cells across the links in 
round-robin fashion. It offers both data-link level load sharing across links, and redundancy. If one of the links in a group fails, 
cells are no longer sent on that one link, but the others are still used. IM A is available only for low-speed links— groups of T1 or 
E1 links. 

Parallel links with network-layer changeover. In this case, a redundant pair of trunks is used, but data-link layer protection is not 
used at all, and all connection changeover takes place at the network layer. IP or PNNI routing is aware of all link failures and 
reacts to them. This is not particularly good for connection-oriented traffic, but works well with IP routing and M PLS. With O SPF 
equal-cost-multipath or similar, O SPF will choose to balance traffic for every route across both linksin a pair of links. T his causes 
a pair of M PLS Label LVCs to beset up for each destination, one per link. If one of the links fails, IP routing will simply divert 
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a. Calculating traffic flows according to IP routing 
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d. Link types, allowing for network-level redundancy 


Figure 6 Network design example - calculating link bandwidths 
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Figure 7 Viewpoints of anATM MPLS network 


traffic onto the other, already-established, LVCs. If VC merge is used, this will require no more M PLS signalling, and could be 
achieved in a second or so. The advantage of this method is that it allows the bandwidth in the ‘backup’ trunk to be used, allowing 
more best-effort traffic to be carried in the network. SONET APS changeovers, as implemented on the non-M SSBU Cisco 
equipment, are a form of parallel link redundancy, but without the capability of equal-cost multipath to set up back-up links. 

Where a redundant link is required, recommendations for use of these modes are as follows: 

¢ IP+ATM networks: these should use Inverse M ultiplexing for redundancy for low-speed trunks, and otherwise use data link-layer 
redundancy. This avoids costly re-routes of the connection-oriented traffic. 

* PureATM M PLS networks: again, these should use Inverse M ultiplexing for redundancy for low-speed trunks. Otherwise, if the 
network uses VC merge, parallel links with network-layer changeover should be used, in order to make the full network capacity 
available for use. Finally, if VC merge is not available, data link-layer redundancy should be used. 
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xX Layer 2 switches are invisible to IP routing 
Cc) An LSC plus a switch is a single IP routing node 


c. Deriving the routing view point 


d. Routing view point 


Figure 7 Viewpoints of an ATM MPLS network (continued) 


3.3 IP Routing in an MPLS Network 
M PLS uses ordinary IP routing protocols—O SPF, IS-IS, etc.—to determine the routes for IP traffic and LVCs. Every LSR runs 


ordinary IP routing protocols in the same way that ordinary IP routers do. An important implication of this is that O SPF (or IS-IS, 

etc.) “sees” an M PLS network as being exactly like an ordinary router network. This is illustrated in Figure 7. It is possible to have 

various viewpoints of an ATM M PLS network: 

* Physical viewpoint: T his viewpoint represents the physical devices and links in a network. An example is shown in Figure 7(a). 

* Functional viewpoint: Where a product has several functions, these can be shown separately. For example, the M GX 8850s in 
Figure 7 each include two separate EdgeLSRs, which are shown separately. In addition, it is useful to think of the PVC switching 
function of an M GX 8850 to be separate from the M PLS switching function. It is sometimes useful to consider the Label Switch 
Controller (LSC) function in an ATM -LSR as being separate from the switching function. This is particularly true if the LSC is 
also acting as an Edge LSR. 
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* Routing view point: T his viewpoint shows the network as it is seen by an IP routing protocol. An example of deriving it is shown 

in Figure 7(c) and (d). 

- Layer 2 PVC switches and PVC switching functions are invisible to IP routing. If a customer site is connected to a router by a 
PVC, then the PVC isa 1-hop direct connection from an IP routing perspective. See for example the sites labelled ‘a’ Figure 7(c), 
and assume that these are connected by PVCs to EdgeLSR ‘y’. Then, in the routing viewpoint, the sites are directly adjacent to 
router ‘y’. 


- A Label Switch Controller and a switch together form a single routing node. 
Using these rules, the routing viewpoint of an M PLS network can be derived. This is shown in Figure 7(d). 


Designing IP routing in an M PLS network is almost exactly the same process as designing IP routing for an ordinary IP network. 

By looking at the routing viewpoint, a network can be divided into Areas, route summarization can be designed, and so on. There 
are several design guides for IP routing on http://w wwin-cons.cisco.com/~dblack/design-guides/. Also, see the book “Internet 

R outing Architectures,” mentioned in theIntroduction. There area small number of routing issues specific to M PLS networks, and 
these are considered next. 


MPLS-Specific IP Routing Issues 

* The interior routing protocol used in M PLS backbones should be either O SPF or IS-IS°. EIGRP can also be used, but it will not 
work with an advanced M PLS-based IP traffic engineering feature called Routing with Resource Reservations (RRR). RRR 
requires a link-state routing protocol, i.e. OSPF or 1S-IS!0. Since EIGRP isa distance-vector routing protocol, it will not work with 
RRR.IGRP or RIP also will work with M PLS but not RRR, and are not recommended. N ote that RRR is sometimes referred to 
loosely as “M PLS Traffic Engineering,” but is actually a specific type of M PLS traffic engineering. 

Use unnumbered IP links where possible. This reduces the number of IP destinations known to the routers, and hence reduces the 
number of LVCs used in the network. This is also discussed under “3.4 Dimensioning M PLS Label VC space” on page 30. 
Route summarization must not be doneat an ATM -LSR. M ultiple OSPF or IS-IS areas can be used in an ATM -M PLS network, 
as shown in Figure 8. An ATM -LSR may be used an OSPF or IS-IS Area Border Router (ABR), but only if no summarization is 
done at the Area Border routers. In Figure 8(c), this means that the address prefixes known in all the areas must be the same. An 
ABR in Figure 8(c) may not, for example, summarize reachability for 1.1.1.0/24, 1.1.2.0/24 and 1.1.3.0/24 with a single route for 
1.1.0.0./16. If route summarization is required in an ATM M PLS network, it must be doneat an ATM EdgeLSR, as shown in 
Figure 8(b). 

The previous rule also applies to Autonomous Systems and BGP 4. An ATM -LSR may not be a BGP Autonomous System 
Boundary Router, but an ATM EdgeLSR may beone. 

Routing with Resource Reservations (RRR) works best in backbones which contain a single O SPF or IS-IS Area. Currently, RRR 
may benot beused in multi-Area networks where the Area Border Routes areATM -LSRs. This restriction will be eased in a later 
version of RRR.14 

Route summarization may not bedonein the interior of an MPLS VPN network. M PLSVPN networks will be discussed in detail 
in a later version of this design guide. The interior of aM PLS network supporting VPNs may have multiple O SPF or IS-IS areas, 
but summarization should not be used. 


9. 1S-ISis currently supported on most Cisco M PLS equipment, but not on the LS1010 and 8540 MSR. 
10. Both the O SPF and IS-IS Working Groups at the IETF are working on extensions to support IP Traffic Engineering, and these capabilities will be used by RRR. 


11. One specific requirement is that a Traffic Engineering (TE) or RRR tunnel must start and end at a device which can “push” and “pop” labels on a packet's label 
stack. Currently this requires that the tunnels start and end at ATM edgeLSRsor other packet-based LSRs. It is theoretically possible to let ATM -LSRs push and pop an 
extra label by carrying the additional label in a range of V Pls. However this has not yet been implemented. Another requirement is that TE/RRR tunnels may not pass 
through Area Border Routers; they must terminate and re-start. Together, these requirements currently mean that it is currently impossibleto useRRR or TE ina network 
where theABRsareATM -LSRs. A futureRRR releasewill allow TE/RRR tunnels to pass through Area Border Routers, and this will permit RRR to beused in networks 
where the ABRs areATM-LSRs. 
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Figure 8 Multiple routing areas and summarization in an ATM MPLS network 
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The restrictions on summarization exist because summarization stops some types of label-switched paths being set up end-to-end. 
For example, assume that an ABR summarizes reachability for 1.1.1.0/24, 1.1.2.0/24 and 1.1.3.0/24 with a single route for 
1.1.0.0/16. Now assume that a packet with IP address 1.1.1.23 arrives with a label for 1.1.0.0/16. The ABR cannot label-switch the 
packet. It must look past the label and examine the IP address to find that the packet should go on to 1.1.1.0/24. Since ATM -LSRs 
cannot examine IP addresses, they may not do IP route summarization?2. 


3.4 Dimensioning MPLS Label VC space 

This design guide has shown how many of the issues of designing M PLS networks are similar to those of designing ordinary IP 
networks. One important exception to this is the dimensioning of MPLS LVC requirements on each link. This design problem is 
illustrated in Figure 9. In order to complete the design of an ATM M PLS network, a sufficient number of VCs must be reserved for 
use as LVCson each link. This can bea problem, as any ATM switch will support only a certain number of active VCs. This is 
particularly important if there are multiple ATM services—M PLS, PNN1I, etc.—sharing the resources of the linksin an IP+ATM 
network. The design problem is to determine what number of LVCs is required. 


How many MPLS LVCs do| 
need to allow on each link? 


Figure 9 Label VC Requirements 


The required number of LVCs depends on: 

* The number of IP destinations in the network 

* The relationship between destinations and LVCs 
¢ Whether VC merge is used 

* The paths chosen by IP routing 


Destinations 

The number of LVCs used in a particular Area of a network depends on the number of IP destination-prefixes known in that area. 

This follows the normal rules for an IP network: 

* The loopback address of all LSRs and other routers in the area is a destination-prefix. 

* The subnet address-prefix of any numbered point-to-point link, or any other subnet, is a destination-prefix. Because of this, it is 
best to use unnumbered links in M PLS networks. 

* Any other address prefixes advertised into the area need to be counted as well!>. If many addresses are summarized into a single 
address at the Area Border Router (or Autonomous System Border Router), then this counts as a single destination-prefix. 

T hese rules are shown in Figure 10. 


12. SomeATM -LSRs, e.g. the BPX 8650, havea limited ability to examineIP addresses by sending the packets to the Edge LSR function in their Label Switch Controller. 
H owever this may be done only for a small minority of the traffic flowing through the ATM -LSR 


13. N ote that in MPLS VPN networks, this does not apply to VPN customers’ addresses. VPN customers’ destination-prefixes are not advertised into the core of the 
network. This is one of the keys to scalability of MPLS VPNs. 
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Figure 10 Destination-prefixes in an MPLS network (or any other IP network) 


LVC’s Used Per Link & VC Merge 

Each ATM EdgeLSR and each Label Switch Controller will ask a neighbouring M PLS node for LVCs for the destination-prefixes 
it knows about. If M PLS Class of Service is used, it may ask for up to four LVCs for each destination-prefix. The requests for LVCs 
flow through thenetwork according to the paths chosen by IP routing. With VC merge, the LV Cs to each destination will be merged 
at each ATM -LSR. This means that on each link, thereis at most oneLVC per destination in the Area. This is shown in Figure 11(a). 
If M PLS Class-of-Service is used, then this is multiplied by the number of classes. If VC mergeis not used, there may be many more 
LV Cs: this is discussed later. 


Design Calculations: Edge LSRs 
For ATM EdgeLSRs, the number of LVCs used per link depends on whether VC merge is being used in the network. Let d bethe 
number of destination-prefixes Known in an area, and c be the number of classes-of-service used in the network. If VC mergeis 
used, then the number of LVCs used per link, | , satisfies 

l<cd. (1) 

If VC merge is not being used in the network, there is also a dependency on the number of LSCs and Edge LSRs in the area. 
There is also a dependency on how many destinations are directly reached through the edge LSR in question. If oe is the number 
of destinations reachable through a particular ATM edgeLSR (this will often equal 1, due to summarization) and the total number 
of ATM EdgeLSRs and LSCsin the area is n, then the number of LVCs used per link satisfies 

I<c(d—d,)+cnd,. (2) 
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Figure 11 LVCs to each destination 


In the particular case where VC merge is not used, and there is one destination-prefix per edge LSR or LSC, and all links are 
unnumbered, and there are no address prefixes from outside the area, a simpler equation applies. These conditions will often apply 
in the core of M PLS networks supporting VPNs, but not using VC M erge. The number of LVCs used per link on the ATM Edge 


LSR in this case is given by: 
l<2cn. 


O ne of these equations is then used to check whether a sufficient number of LVCs is available on the equipment, as shown in 


Table 6. Table 7 shows the LVC capacity of Cisco ATM EdgeLSR interfaces. 


Table 6 Checking the LVC limits of Edge LSR 


Device Situation Key Parameter Check Against 
Edge LSR The network uses VC merge Number of active VCs supported per ATM link. | Equation (1) 
Edge LSR The network does not use VC merge, there is one destination-prefix per Number of active VCs supported per ATM link. | Equation (3) 
LSR or edge LSR, all links are unnumbered, and there are no out-of-area 
routes. 
Edge LSR The network does not use VC merge, all other situations. Number of active VCs supported per ATM link. | Equation (2) 
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Table 7 Cisco ATM Edge LSRs and LVC capacity 


Number of 
; Interface active LVCs 
Device hardware supported Notes 
3600 NM-1A ATM 1024 
Network Modules 
4700 NP-1A ATM 1023 
Network 
Processor 
Module 


7200, 7500 PA-Alorstandard 2048 
ATM port adaptor 


Catalyst PA-A3 ATM port 4096 

5500, 7200, adaptor. 

7500 

6400 Node Route 2048 Capacity is reduced by 1 LVC for each active PVC which terminates on 

Processor (NRP) the NRP. 

MGX 8850 Route Processor 4096 Capacity is reduced by 1 LVC for each active PVC which terminates on 

IP+ATM M odule (RPM ) the RPM. In addition, the PXM is limited to 16k LVCs. This is unlikely to 

switch bea problem unless more than 3 RPMs are used in an M GX 8850 shelf. 

12000 4x0C3 ATM Line 2047 [Needs The 2047 active VCs are shared between all four ports. N etwork 

series Card confirmation] Capacity is reduced by 1 destination-prefix for every second and 

routers subsequent route chosen for each destination according to 
equal-cost multipath routing, if the extra route(s) are on the same 
card. 

12000 1x0C12 ATM line 2047 [Needs 

series card confirmation] 

routers 


Edge LSRs: Worked Examples 


Q. Consider a network where VC merge is being used and one class of service is being used. If the edge LSRs are all 7200-series 
routers with PA-A3 port adaptors, then how many IP destination prefixes can safely be supported in the area? 

A. VC merge is being used, so Table 6 indicates that Equation (1) should be used. One class of service is being used, so c = 1. 
Table 7 states that the PA-A3 port adaptor supports 4096 LVCs, so | = 4096. Substituting these into Equation (1) gives 


4096 <1d 


Or d>4096. This means that 4096 destination-prefixes are guaranteed to be supported within the area, provided that the 
ATM -LSRs do not impose a tighter limit (this discussion has considered only the Edge LSRs). 


Q. Consider a network where VC merge is not used, and 4 classes of service are being used. This network is the core of an M PLS 
VPN service, and thereis one destination-prefix per LSR or EdgeLSR. All links are unnumbered. N o out-of-area routes are injected 
into the interior routing protocol. The edgeLSRs are 7200 and 7500-series routers with a mixture of PA-Al and PA-A3 ATM port 
adaptors. W hat is the largest number of LSRs which can be used, if the network consists of a single Area? Assume that the 

ATM -LSRs support a sufficiently large number of LVCs. 

A. According to theconditions given, Table 6 indicates that Equation (3) should beused. Four classes of service are used, Soc = 4. 
The Table 7 shows ATM interfaces each support 2048 or 4096 LVCs. The interfaces with 2048 LVCs give a tighter restriction, so 
use | = 2048. Substituting these into Equation (3) gives 


2048 < 2(4)n 
Or n>256. This means that a maximum of 256 LSRs (edge LSRs or ATM -LSRs5) may be used in the area, provided that the IP 
routing protocol supports that many routers in an area. 
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Q. Consider a network where VC merge is not used, and 4 classes of service are being used. This network is the core of an M PLS 
VPN service, and there is one destination-prefix per ATM -LSR or Edge LSR. All links are unnumbered. The network has multiple 
areas, and there are at most 100 ATM -LSRs and LSRs in each area. All the edge LSRs’ ATM port adaptors are PA-A3. Assume that 
the ATM -LSRs support a sufficiently large number of LVCs. How many LSRs can be used in the entire network? 

A. Thereare multiple areas, and so there will be out-of-area routes in each area. Table 6 indicates that Equation (2) should be used 
in this case. Four classes of service are used, So c = 4. Table7 gives! = 4096. Thereareat most 100 ATM -LSRsand LSRsin each 
area, so usen = 100. 


O bserve that there is one route per edge LSR. In the worse case, all out-of-area routes are accessed through a single LSR — this 
concentrates the LVC requirements on the link(s) to that single LSR. In this case, d, = (d- 100). Substituting these into 
Equation (2) gives 


4096 < 4(100) + 4(100)(d - 100) 


(d — 100) = 3696/7400 


or d>109. This means that only 109 LSRs can be used in the network. By comparison with the previous example, we can see that 
using multiple areas can have major disadvantages in ATM M PLS networks without VC merge!’. 


Design Calculations: ATM -LSRs With VC M erge 
With VC merge, the LVCs to each destination will be merged at each ATM -LSR. This means that there is at most one LVC per 


destination on each link, as shown in Figure 11(a). If M PLS Class-of-Service is used, then this is multiplied by the number of classes. 
If d is the number of destination-prefixes known in an area, and c is the number of classes-of-service used, then the number of 
LV Cs used per link, |, satisfies 
heed: (4) 
Another important issuein switches which support VC mergeis the number of LVCs which must be merged together in the switch, 
say m. This depends on the number of links into the switch, say k . The limit is: 
m<cd(k—1). (5) 
T hese equations is then used to check whether a sufficient number of LVCsis available on the equipment, as shown in Table 8. 
Both equations will need to be checked. Table 9 shows the limits of Cisco ATM -LSRs with VC merge. 


Table 8 Checking the LVC limits of ATM -LSRs with VC Merge 


Device Key Parameters Check Against 
ATM-LSR with VC 1. Number of active VCs supported per ATM link. Equation (4) 
Merge 2. Number of merging LVCs supported per switch, or per port Equation (5) 

card, whichever is applicable to the switch architecture. 


a. In general, a per-switch limit will apply to shared-memory switches suchas the LS1010 or 8540 M SR, and a per portcard 
imit will apply to crossbar switches such as the BPX 8650 or MGX 8800 with PXM -45. 


ATM-LSRs With VC M erge: Worked Examples 


Q. A network uses BPX 8650 ATM -LSRs with VC merge. Two classes of service are used. Each BPX 8650 has 4x1-port 
OC12/STM -4 BXM cards, with each port used to link to another ATM -LSR or edgeLSR. What limit do these ATM -LSRs put on 
the number of IP-destination-prefixes which can be supported inside an area? 


14. These examples indicated that ATM -LSRs without VC merge typically cannot be used in networks of larger than a few hundred nodes. An alternative which works 
around these limitations is to use the same switches, but to use M PLS-over-PVCs instead of ATM MPLS. Thisis discussed further in “4.2 N etworks Using 
M PLS-over-PVCs” on page 41. 
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Table 9 Cisco ATM -LSRs and LVC capacity, if VC merge is used 


Device Interface Hardware Number of active LVCs supported? | Number of active merging LVCs supported® 


LS1010 Any ATM port hardware. 4096 per OC3 port, 16k per OC12 port, 16k? 64k per switch, 
per 0C48 port [needs confirmation] 
6400 Any ATM port hardware 4096 per OC3 port, 16k per OC12 port, 16k? 256k per switch 
per 0C48 port [needs confirmation] 
8540 MSR Any ATM port adaptors 4096 per OC3 port, 16k per OC12 port, 16k? 256k per switch 
per 0C48 port [needs confirmation] 
BPX 8650 BXM -E cards 32k per BXM, shared amongst up to 12 32k per BXM, with a maximum of 16k per port on OC3 
or 8680 interfaces BXM cards and 2x0C12 BXM cards. 13/E3 BXM cards 
and 1x0C12 BXM cards have a limit of 32k per port. 
MGX 8800 AXSM cards 128k per AXSM, shared amongst up to 16 128k per AXSM 
with interfaces [Needs confirmation.] 
PXM-45 


@ 


. These numbers are maximums, and the actual limits will be dependent on configurations. In a BPX 8650, for example, the actual number of active LVCs 
supported per link must be down-rated by a minimum of 270 lines per interface if AutoRoute is enabled on that interface. On all switches, the VC space reserved 
or PVCs, SVCs, etc. must be subtracted from the available VC space. 


A. Table 8 shows that both Equation (4) and Equation (5) need to be checked. Two classes of service are used, So c = 2. Each 
switch has four ports, so k = 4.Lookingup the BPX 8650 in Table 9 shows that BX M cards support 32k active LVCs. In this case, 
each BXM card has one port, so each link supports 32k LVCs, or | = 32768. Table 9 shows that 32k LVCs can be merged into a 
1-portO0C12 BXM card, so m = 32768. 


Substituting these parameters into Equation (4) gives 
32768 < 2d 
or d>16k . Substituting the parameters into Equation (5) gives 
32768 < 2d(4-1) 


or d>5461.Thelimit from Equation (5) is tighter, which means that the limit imposed by the ATM -LSRs is 5461 
destination-prefixes in the area. (The edge LSRs might impose a tighter limit.) 


Q. A network uses 8540 MSR ATM -LSRs with VC merge. Four classes of service are used. Each 8540 M SR has 8xOC3/STM -1 
with each port used to link to another ATM -LSR or edge LSR. What limit do these ATM -LSRs put on the number of 
IP-destination-prefixes which can be supported inside an area? 

A. Table 8 shows that both Equation (4) and Equation (5) need to be checked. Four classes of service are used, So c = 4. Each 
switch has eight ports, so k = 8. Looking up the 8540 M SR in Table 9 shows that OC3 port cards support 4096 LVCs, or 

| = 4096. Similarly, Table 9 shows that the 8540 M SR supports 256k merging VCs, som = 262144. 


Substituting these parameters into Equation (4) gives 
4096 < 4d 
or d>1024. Substituting the parameters into Equation (5) gives 
262144 < 4d(8- 1) 


or d>9362.Thelimit from Equation (4) is tighter, which means that the limit imposed by the ATM -LSRs is 1024 
destination-prefixes in the area. (The edge LSRs might impose a tighter limit.) 


Q. A network usesBPX 8650 ATM -LSRs with VC merge. Four classes of service are used. Each BPX 8650 has 8 ports, on 2x4-port 
OC3/STM-1BXM cards, with each port used to link to another ATM -LSR or edge LSR. What limit do these AT M -LSRs put on the 
number of IP-destination-prefixes which can be supported inside an area? 
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A. Table 8 shows that both Equation (4) and Equation (5) need to be checked. Four classes of service are used, So c = 4. Each 
switch has eight ports, so k = 8. Looking up the BPX 8650 in Table 9 shows that BXM cards support 32k active LVCs. In this 
case, each BX M card has four ports, so wecan assume that each link supports 32k/4 LVCs, or | = 8192. Similarly, Table 9 shows 
that 4x0 C3 BXM cards can support 32k merging VCs, with a maximum of 16k per port. The worst case is when all LVCs try to 
merge into the same port, so useso m = 16384. 


Substituting these parameters into Equation (4) gives 
8192 <4d 
or d>2048. Substituting the parameters into Equation (5) gives 


16384 < 4d(8 - 1) 


or d>585.Thelimit from Equation (5) is tighter, which means that thelimit imposed by theAT M -LSRsis 585 destination-prefixes 
in the area. (The edge LSRs might impose a tighter limit.) 


Design Calculations: ATM -LSRs Without VC Merge 
Without VC merge, there may be many VCs per destination on each link, as shown in Figure 11(b). If the total number of ATM 


Edge LSRs and LSCsin the area is n, then there may beup to c(n—1) LVCs per destination on each link!°. The number of LVCs 
used per link will then satisfy 

l<cd(n—1). (6) 
A tighter limit applies in the particular case where VC merge is not used, and there is one destination-prefix per edgeLSR or LSC, 
and all links are unnumbered, and there are no address prefixes from outside the area. T hese conditions will often apply in the core 
of M PLS networks supporting VPNs, but not using VC M erge. The number of LVCs used in this case is given by: 

l<c(n2/2). (7) 

One of these equations is then used to check whether a sufficient number of LVCs is available on the equipment, as shown in 

Table 10. Table 11 shows the limits of Cisco ATM -LSRs without VC merge capability. 


Table 10 Checking the LVC limits of ATM -LSRs without VC M erge 


Device Situation Key Parameter Check Against 
ATM-LSR without VC There is one destination-prefix per LSR or edge LSR, all links are Number of active VCs supported per ATM link. | Equation (7) 

Merge unnumbered, and there are no out-of-area routes. 

ATM-LSR without VC All other situations. Number of active VCs supported per ATM link. | Equation (6) 
Merge 


Table 11 Cisco ATM-LSRs and LVC Capacity, if VC merge is not used 


Device Interface Hardware Number of active LVCs supported per link? 


BPX 8650 Older BXM cards (or pre-9.3.x software) 16k per BXM , shared amongst up to 12 interfaces 


a. These numbers are maximums, and the actual limits will be dependent on configurations. In a BPX 8650, for example, the 
actual number of active LVCs supported per link must be down-rated by a minimum of 270 lines per interface if AutoRoute is 
enabled on that interface. On all switches, the VC space reserved for PVCs, SVCs, etc. must be subtracted from the available 
VC space. 


15. This assumes that all LSRs in the area which are performing label switching (as opposed to label edge) function are AT M -LSRs. In other words, it assumes that the 
only router-based LSRs are at the edge of the area. 
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ATM-LSRs With VC M erge: Worked Examples 


Q. A network uses BPX 8650 ATM -LSRs without VC merge. One class of service is used. Each BPX 8650 has 2x4-port 
OC3/STM -1 BXM cards, with each port used to link to another ATM -LSR or edgeLSR. There is one destination-prefix per LSR 
or edge LSR. All linksin the area are unnumbered, and there are no out-of-area routes known. What limit do these ATM -LSRs put 
on thenumber of LSRs or edge LSRs which can be supported inside an area? 

A. Table 10 shows that Equation (7) should be checked in this case. O ne class of service is used, so c = 1. Looking up the BPX 
8650 in Table 9 shows that BX M cards support 16k active LVCs. In this case, each BX M card has four ports, so each link supports 
16k/4 LVCs, or | = 4096. 


Substituting these parameters into Equation (7) gives 


4096 <1(n2/2) 


2 
(n° => 8192) 
or n>90. This means that the limit imposed by the ATM -LSRsis 90 LSRs or edgeLSRs!°, 


Q. A network uses BPX 8650 ATM -LSRs without VC merge. Two classes of service are used. Each BPX 8650 has 4x1-port 
OC12/STM -4 BXM cards, with each port used to link to another ATM -LSR or edgeLSR. What limit do these ATM -LSRs put on 
the number of IP-destination-prefixes which can be supported inside an area? 

A. No information is given on the relationship between devices and routes, so Table 10 shows that both Equation (6) should be 
checked. Two classes of service are used, So c = 2. Looking up the BPX 8650 in Table 9 showsthat BXM cards support 16k active 
LVCs. In this case, each BX M card has one port, so each link supports 16k LVCs, or | = 16384. 


Substituting these parameters into Equation (6) gives 
16384 < 2d(n-1) 
d(n—1)>8192 
n, the number of LSRs in the area, has not been given, so this question cannot be explicitly answered as stated. H owever, if ait is 
assumed that (say) n = 50, this would giveand indicative value d > 167. In other words, the number of destination prefixes which 


may be supported depends on the number of LSRsin the area with, for example, a limit of 167 destination-prefixes if there are 50 
LSRs in the area. 


Note on Internet Routing Tables 
Thelimits on destination-prefixes indicated in the previous examples are much smaller than the size of the Internet backbonerouting 
table, which is about 70,000 routes. Despite this, ATM M PLS can still be used in networks with full Internet routing, by use of an 
M PLS feature known as BGP Next-H op Labelling. This feature allows BGP Autonomous System Boundary Routers (ASBRs) to 
exchange the full Internet routing table with each other by way of BGP, while readvertising only a limited subset of these addresses 
(or noneat all) into the Interior routing protocol (O SPF or ISIS) Area(s) through which they are connected. Since only a limited set 
of destination-prefixes is known on OSPF or IS-IS in the M PLS network, the limits discussed here are sufficient even though they 
are much smaller than the Internet routing table. 

Cisco M PLS Virtual Private N etworks (VPNs) extend the BGP N ext-H op Labelling technique to deal with address from many 
different customers’ VPN s. 


16. These examples indicated that ATM -LSRs without VC merge typically cannot be used in networks of larger than a few hundred nodes. An alternative which works 
around these limitations is to use the same switches, but to use M PLS-over-PVCs instead of ATM MPLS. This is discussed further in “4.2 N etworks Using 
M PLS-over-PVCs” on page 41. 
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Traffic Engineering 
The limits shown in Equations (4) to (7) apply when M PLS Traffic Engineering is not being used. If Traffic Engineering is being 
used, then one LVC will be used for each Traffic Engineering Tunnel on each link, in addition to the limits shown above. 


VP Tunnels 
VP Tunnels are described in “4. M igration of M PLS into Traditional ATM Networks” on page 40. T hey involve several logical links 


terminating on asingle physical interfaceon an LSR or ATM -LSR. When VP Tunnels are terminate on an interface, the LVCs on all 
VP Tunnels must be taken into account. For example, if 4 VP Tunnels terminate on a logical interface which supports 4000 LVCs, 
then an average of only 1000 LVCswill be available per VP Tunnel. This issue is considered in more detail in “4.6 VP Tunnels and 
LVC Usage” on page 53. 


Alternative Calculations 
The limits shown in Equations (4) to (7) can be quite ‘loose’. The number of LV Cs actually used on a particular link may be much 


less than these limits suggest, particularly if VC merge is not being used. H owever it is difficult to calculate exactly how many will 
be required. This depends on the exact shape and state of the network, and the exact paths chosen by IP routing. If this can be 
analyzed, taking account of such things as failed links and multipath routing, then lesser numbers of LVCs could be safely reserved 
on each link. This would be a quite complex process. In any case, the limits shown above will be safe. 


The Last Word 
Josh Gahm, who wrote much of the LSC code, said this about running out of space for LVCs: 


“a network that has run out of cross-connects is basically non-functioning. There really isn't any way to control which [LVCs] 
get created and which don't. This may result in permanent black holes and/or [overloaded LSCs]. The network needs to be 
designed from the start such that it won't run out of connections.1”” 
In other words, it is very important to design the network and allocate switch resources so that a sufficient number of LVCs are 
available on each link. 


3.5 On-going Network Design 
N etwork design is an on-going process. Once an ATM -M PLS network is deployed, on-going design activities are required to: 


* Verify the assumptions used in the initial design 
¢ Adjust the network as new customers and PoPs are added. 
The on-going process will involve the following steps: 


1. Measure actual PoP and link traffic, and compare against 
- The predicted traffic 
- Thelink capacities 
2. Based on the comparison between predicted and actual traffic, some modelling assumptions might be changed. For example, the 
traffic distribution across nodes might be different to that which was initially predicted. R eview the initial design and 
dimensioning if the modelling assumptions are changed. Lower and higher-bandwidth links might be required. 
3. Ascustomers are added, and as traffic increases, review the initial design to: 
- Add new edge LSRs to PoPs 
- Add new links to the network 


17. Two different behaviors may occur when a link runs out of LVCs, depending on the type of traffic. The first case applies to ordinary IP traffic which would otherwise 
be carried with a single M PLS label. In this case, an edge LSR will deal with LVC set-up failure by sending the ordinary IP traffic on the default (0,32) LVCs, which are 
otherwise used for TDP/LDP signalling. This traffic will then be forwarded around the resource-starved links by the LSC processors, and not the ATM switch fabrics. 
This will not affect network stability, provided that M PLS Class of Service is used to give precedence to routing and TDP/LDP traffic. H owever if large amounts of user 
traffic are sent on the (0,32) LVCs, then this user traffic will experience poor performance when it exceeds the LSCs’ packet forwarding capacity. 

The second case applies to advanced M PLS services such as V PN s. In this case, the packets which would otherwise use the failed paths are discarded. For these services, 
it is impossible to correctly deliver a packet unless it is fully labelled. This discarding of packets to certain destinations results in routing “black holes”. 

In case of LVC starvation, it is usually impossible to predict which LVCs get created and which don’t. This is because LVCs are created roughly in the order in which 
routing converges. Thisis, in turn, dependent on random factors such as which links fail, and the exact timing of link failures compared to routing protocol update timers. 
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- Adjust routing 
- Check for sufficient LVC allocation on links 
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4, Migration of MPLS into Traditional ATM Networks 


4.1 Hybrid ATM Netw orks 

Sometimes it is necessary to connect ATM M PLS devices over traditional ATM equipment, in a ‘hybrid ATM network’. This is 
undesirable, as many of the disadvantages of traditional IP-over-ATM schemes, as discussed in “1.5 M PLS and Other |P-over-AT M 
Schemes” on page 7, are re-introduced in hybrid ATM networks. In particular: 

* Hybrid ATM networks are inefficient at providing IP services such as multicast. 

¢ Hybrid ATM networks have routing scaling problems similar to those of IP-over-PVC networks. 

* Hybrid ATM networks are more difficult to manage than pure M PLS networks. 

H owever hybrid ATM networks can support M PLS services such as Virtual Private N etworks, and may be necessary as a migration 
step while introducing a full M PLS network. 

H ybrid networks are not the same as IP+ATM networks. In |IP+ATM networks, all ATM switches have both M PLS capability 
and traditional ATM switch capability. In hybrid networks, some of the switches do not haveM PLS capability. Someof the switches 
in hybrid networks may have M PLS capability, and these particular switches may be |P+ATM switches. 

There aretwo broad ways of implementing hybrid ATM networks: M PLS-over-PVCs and VP Tunnels. These are illustrated in 
Figure 12. M PLS-over-PV Cs can be used only to connect packet-based M PLS devices. It may not be used to connect ATM Label 
Switch R outers (ATM -LSRs). M PLS-over-PV Cs connects packet-based Label Switch R outers (LSRs) by way of PVCs over a 
traditional ATM network. Therouters send M PLS packets to each other, with labels explicitly encapsulated along with the IP 
packet. This is called ‘packet-based labelling’, as the M PLS label is applied to a whole packet, as opposed to individual cells. When 
packet-based labelling is used over PVCs, packets with many different labels are sent in the same PVC. This differs from ATM 
M PLS, wherea each different label is represented by a different VC, known asa ‘Label VC’ (LVC). Packet-based labelling over PVCs 
is virtually identical to the case where M PLS Label Switch Routers (LSRs) are connected by links such as Packet-over-SONET, 
Packet-over-SDH, or any other point-to-point links. M PLS-over-PV Cs does not useATM M PLS at all. This means that service 
providers must continue to provision and manage PVCs on a scale equal to the traditional IP over ATM approach. H owever 
M PLS-over-PV Cs does have the advantages of simplicity, and of using more stable |OS code than VP shaping. 


ATM link supporting full MPLS 


ATM Label Switch Router 


PVP used as a VP Tunnel 


Non-MPLS ATM Network 


ass ATM Edge LSR 


Non-MPLS ATM Switches PVC supporting MPLS traffic 


Figure 12 VP Tunnels and MPLS-over-PVCs 


VP Tunnels 

The remaining method of implementing hybrid ATM networks is the use of Virtual Path (VP) Tunnels. ATM M PLS normally 
involves labelling IP packets by putting them in different VCsin the same ATM trunk. Each different VC on the trunk represents a 
different label value. VP Tunnels area form of ‘Virtual Trunk’. ATM M PLS devices treat them almost identically to a physical trunk: 
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each different VC within the VP represents a different label value. The differenceis that the VP Tunnel is not a physical trunk linking 

two adjacent ATM -LSRs. The VP Tunnel is a Permanent Virtual Path Connection (PVP) which connects AT M -LSRs by way of 

traditional ATM switches. VP Tunnels may also connect ATM EdgeLSRsto ATM-LSRs, or connect ATM EdgeLSRs to each other. 
Configurations for hybrid networks, and consequent equipment requirements, are considered in the following sections. 


4,2 Networks Using MPLS-over-PVCs 

Thesimplest network structure using M PLS-over-PV Cs is the full mesh shown in Figure 13(a). O peration of IP routing protocols in 
an M PLS network of this structure leads to the same scalability issues as for traditional IP-over-ATM networks of similar 
structure— refer to the discussion in “1.5 M PLS and Other IP-over-ATM Schemes” on page 7. O nesolution to this is to use a partial 
mesh between the routers, but this would lead to the use of inefficient, multi-hop routes. Another alternative is to add extra ATM 
EdgeLSRs, asshown in Figure 13(b), or possibly a redundant pair of them. TheextraATM EdgeLSRsreducethesize of the meshes, 
but often won't have customer-facing interfaces. N ote that the performance requirements on the extra LSRs will be quite high, as 
they will carry a large part of the network traffic. There is no direct way of using ATM -LSRsin a network using M PLS-over-PVCs. 


¢ 
w / 
Ae - i! 
“ ) / % 
\ ] | ¢ \ 
AS : | au 
~.) ( (ae 
| \ — a . ) 
ji a- 4a ASS I ) 
a , eed 
y 
x 


b. Adding an extra packet-based 
LSR for scalability 


c. Possible migration towards a d. Possible final stage: a 
packet-based MPLS network packet-based MPLS network 
- +" a 2) Non-MPLS ATM network and a PVC ATM Edge LSR or 
~—- ~ other packet-based LSR 


— MPLS link 


Figure 13. ATM MPLS networks using M PLS-over-PVCs 
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Some providers might prefer to build an infrastructure for their M PLS traffic which is separate from their traditional ATM 
network, in a ‘dual backbone’ configuration. This M PLS network might useATM M PLS. Alternatively, it might use packet-based 
M PLS, with packet-based LSRs and links such as PPP-over-SDH . Such a packet-based M PLS network might use M PLS-over-PVCs 
as a transitional stage, allowing a traditional ATM network to be used to carry M PLS traffic in the early stages of introduction of 
the packet-based M PLS network. In the course of growing this network, the M PLS-over-PV C links might be replaced with physical 
links. This possible future migration is shown in Figure 13(c) and (d). 
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Figure 14 PVC-based migration to dual backbones: another view 


Another view point of migration to dual backbones is shown in Figure 14. In this example, the Edge LSRs arepart of IP+ATM edge 
devices. T he stages of migration in Figure 14 arethesameasin Figure 13, but Figure 14 shows moreclearly how the migration leads 
to the building of a new packet-based backbone alongside the existing traditional ATM backbone. If IP+ATM edge switches are 
used, then the same switches can connect to both the traditional ATM backbone and the M PLS backbone. Figure 14 shows an 
example where the new backbone uses packet-based M PLS, but it is also possible to use ATM M PLSin the second backbone. 
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Equipment for M PLS-over-PVCs 

The core of an M PLS-over-PV Cs network is a plain old ATM network which needs only to support PVCs. Virtually any ATM 
network can be used. Virtually any M PLS release on Cisco Edge LSRs can be used with M PLS-over-PV Cs. N ote that if 

M PLS-over-PV Cs is being used, a mode of M PLS Class-of-Service using PV Cs instead of LVCs, and based on Project Paris, may also 
be used. The “Recommended Configuration” column in Table 12 is for M PLS-over-PV Cs with Class-of-Service support. Earlier 
releases (e.g. 11.1C T(23)) without Class-of-Service support may also be used. 


Table 12 Cisco ATM Edge LSRs and M PLS-over-PVCs 


PVC traffic types 
supported with 
recommended 
ae configuration 
Minimum Recommended 
Device configuration configuration CBR? VBR ABR 
Catalyst Route Switch Module PA-A3 ATM port adaptor, "4 V "4 
5500 (RSM) with PA-A1ATM Port OS 12.0T(5), or later. 
Adaptor. 10S 
11.3(5)WA4(8b). 
6400 Node Route Processor. 10S inimum or later. Vv V "4 
12.0(6)DC 
7200 series PA-A1 port adaptor, 10S PA-A3 ATM port adaptor, Vv "4 Vv 
routers 11.1CT(20) or later. OS 12.0T(5) or 12.0T(6), or 
ater. 
7500 series PA-A1 port adaptor, 10S PA-A3 ATM port adaptor, Vv Vv Vv 
routers 11.1CT(20) or later. OS 12.0T(5) or 12.0T(6), or 
ater. 
M GX 8850 Route Processor Module inimum or later. v (Va (VA 
IP+ATM (RPM). 10S 12.0T(5.0.2). 
switch 
12000 10S 12.05(5) inimum or later "4 x x 
series 
routers 
a. This means that the shaper will shape to a peak rate. The resulting shaped cell stream is guaranteed to fit within a 
CBR PVC, and does comply with the standard policing parameters for CBR. Note, however, that the actual rate of the 


stream varies up to the peak rate. In some Cisco product literature, this has been referred to incorrectly as ‘VBR’. 


Using Traditional ATM Switches for Backhaul 

Figure 15 shows another type of network which may occur in practice. Figure 15 shows a network which is not strictly a hybrid 
network, as the M PLS traffic is carried only across M PLS devices, and not on traditional ATM switches. In this network, the 
traditional ATM switches are used to connect the CPE into the edgeLSRs, by way of ‘backhaul’ PVCs. These backhaul PV Cs carry 
traditional |P-over-ATM , and do not carry M PLS traffic. This was briefly discussed “2.1 Structures for M PLS N etworks” on page 9, 
and shown in Figure 2(f). Other variations are possible. For example, a network similar to Figure 15 could be used, but with VP 
tunnels or M PLS-over-PV Cs in the network core, as well as the backhaul PVCs. 


4.3 Networks Using VP Tunnels 

A simple way of using VP Tunnels is use them to connect ATM Edge LSRs without using any ATM -LSRs at all in the network, as 
illustrated Figure 16(a). This means that all M PLS packets are carried over VP Tunnels, and no label switching actually occurs in 
the network. The ATM part of the network consists entirely of traditional ATM switches. M ore commonly, some switches in the 
ATM network will support M PLS, and some won't. VP tunnels may be used to connect ATM -LSRs to ATM-LSRs, or to connect 
ATM EdgeLSRs to ATM -LSRs, as well as connecting ATM EdgeLSRs to each other. This is shown in Figure 16(b). 


Migration to Full MPLS 
Figure 16(a), (b) and (c) show a possible migration process for introducing M PLS to a traditional ATM network: 
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Figure 15 Network using backhaul PVCs 


¢ MPLS edge routers (‘ATM EdgeLSRs’) are added around the edge of a traditional ATM network; alternatively, M PLS function 
may be added to existing routers. This enables M PLS VPNs as well as leading to the next steps. 

* Next, M PLS function is added to someATM switches, or extra AT M -LSRs areadded to the network. This reduces the number of 
VP Tunnels required, and starts to reduce some of the scalability problems of hybrid networks. 

* More ATM -LSRS are added, which further reduces the number of VP tunnels, and starts to introduce native ATM M PLS links 
this is shown in Figure 16(c). This step naturally leads to the final one. 

¢ Ultimately, all ATM switches areATM -LSRs, and no VP Tunnels are used at all. The full network runsATM MPLS, and hasnone 
of the disadvantages of hybrid networks. This is illustrated in Figure 16(d). 


Other Variations of Hybrid N etworks 
M PLS devices and traditional ATM switches can be combined in many different ways. Figure 16 shows a few examples. Figure 17 


shows some other hybrid network structures which may occur. M any other hybrid network structures are possible An ATM MPLS 
network must include edge L SRs, but may use any nearly any combination of zero or moreATM -LSRs and zero or moretraditional 
ATM switches with VP tunnels. 


4.4 Requirements to Support VP Tunnels 
VP Tunnels are implemented using PVPs, or alternatively PN NI Soft Permanent Virtual Paths (SPV Ps). Use of VP Tunnels will 


generally require the PVPs or SPVPs to be CBR, as shaping a VP to other than a peak rate is difficult. 


Edge LSRs 
ATM EdgeLSRs must meet the following requirements in order to support VP Tunnels: 
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a. Hybrid network using only VP Tunnels b. Adding an ATM-LSR 


c. Further simplification by d. Full ATM MPLS network 
adding more ATM -LSRs 
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Figure 16 ATM MPLS networks using VP Tunnels 


* They must be able to support distinct logical M PLS links in different sub-interfaces of a physical ATM link. If this requirement is 
not met, each EdgeLSR in would require a separate physical ATM interface for each VP Tunnel. This would create scalability 
problems. For example, each Edge LSR in Figure 16(a) would require four ATM links, whereas each Edge LLSR in Figure 16(d) 
would require only one (or two if it was dual-homed). If a particular VP tunnel uses a VPI x at the Edge LSR, then the LDP 
signalling VC for the VP Tunnel must be within x. It may have V Pl=x, VC1=32, instead of the normal default VPI=0, VCI=32 for 
LDP signalling. 

* They must support shaping of packet traffic entering the VP tunnel to the traffic parameters (e.g. a CBR peak rate) of the VP 
tunnel. It is highly preferable that this be done using Early Packet Discard (EPD), or at least Partial Packet Discard (PPD). The 
alternative to EPD or PPD is Cell Loss M ultiplication, which results in a horrible congestion collapse characteristic for the VP 


Tunnel. 
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a.M PLS core, with VP tunnels to edge LSRs b.M PLS switching near edge, with traditional ATM core 
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Figure 17 Other hybrid network examples 


Cell loss multiplication occurs when a whole packet must be discarded at reassembly because only one cell was lost in transit. For 
example, assume that during a certain timeframe it is necessary to discard 100 cells at ingress to a VP tunnel due to overload. 
Assume that the mean packet size is 5 cells. Without EPD or PPD, the 100 cells may well be dropped from 100 different packets. 
This would lead to the 100 packets being discarded at reassembly. This would cause an effective loss of 500 cells when only 100 
were discarded. This is cell loss multiplication. EPD ensures that if any cell in a packet is dropped, all cells are dropped. T his will 
concentrate the cell loss in around 20 packets?, avoiding cell loss multiplication entirely. PPD ensures that once one cell in a packet 
is dropped, all remaining cells in the packet will be dropped, although some will still have been accepted. This will concentrate 
the cell loss in about 40 packets*, which is a reduction but not elimination of cell loss multiplication. 


Capabilities of Cisco Edge LSRs are shown in Table 13. 


1. This is an approximation, based on the assumption that the queueing behavior at ingress to the VP tunnd is such that the queue is never completely empty. It is also 
assumed that the number of cells to be dropped (100) is invariant with the discard policy. 


2. In addition to the previous assumptions, this approximation assumes that packet loss first occurs at a random cell in each packet. The exact number depends on the 
exact traffic characteristics, the exact distribution of packet sizes, and the exact queueing behavior during the overload. 
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Table 13 Cisco ATM Edge LSRs and VP Tunnels 
ae Smart 

Minimum software & | VP Tunnel traffic discard for 
hardw are for edge types supported VPs 
LSR support for VP 

Device Tunnels CBR? | VBR | ABR | EPD | PPD | Capabilities & limitations 

Catalyst 5500 10S 12.0T(5), Route Switch "4 x x "4 256 VP Tunnels are supported on a PA-A3 card. The total number of 
M odule (RSM) with PA-A3 LVCs is limited to 4096, irrespective of the number of VP Tunnels 
ATM port adaptor. used. 

6400 Any software supporting Vv x x x x The 6400 will also act as an ATM switch and ATM-LSR. 

PLS The Edge LSR function in the 6400 can support VP tunnels only in 
combination with the 6400's ATM -LSR function, as the edge LSR(s) 
rely on the ATM -LSR to perform VP shaping. See “Edge LSRs 

ntegrated into Edge Switches” on page 50 
7200 series OS 12.0T(4), PA-A3 ATM v x x ("A 256 VP Tunnels are supported on a PA-A3 card. The total number of 
routers port adaptor LVCs is limited to 4096, irrespective of the number of VP Tunnels 
used. 
7500 series OS 12.0T(4), PA-A3 ATM v x x Vv 256 VP Tunnels are supported on a PA-A3 card. The total number of 
routers port adaptor LVCs is limited to 4096, irrespective of the number of VP Tunnels 
used. 
M GX 8850 OS 12.0T(5.0.2) "4 x x Vv 256 VP Tunnels are supported on an RPM card. The total number of 
IP+ATM LVCs is limited to 4096, irrespective of the number of VP Tunnels 
switch used. 
Note that the M GX 8850 can also act as an ATM switch and 
ATM-LSR. 
12000 series VP Tunnels are not x x x x x VP Tunnels are not currently supported due to lack of VP shaping 
routers currently supported. capability. This will be fixed in later port card hardware. 
a. This means that the shaper will shape to a peak rate. The resulting shaped cell stream is guaranteed to fit withina a CBR PVP. 


Traditional ATM Switches 
Theswitches in the ATM clouds must support PVP or SPVP connections with ATM Forum or ITU traffic management types which 


match those used on the EdgeL SR s.C apabilities of Cisco switches are shown in the following table. N otethat this table applies only 
when the switches are being used as traditional ATM switches. M ost of the switches shown below can also acts as ATM -LSRs. If 
non-Cisco ATM switches are used, refer to the manufacturer’s documentation for those switches. 


Table 14 Using Cisco ATM switches as traditional ATM switches. 


PVP Traffic Types 
Supported 
Device CBR | VBR | ABR | Capabilities & Limitations 
LS1010 & Vv "4 Vv Supports up to 256 PVPs per trunk, or 8k per switch. 
Catalyst 5500 
6400 "4 "4 "4 Supports up to 256 PVPs per trunk, or 8k per switch 
IGX 8400 Vv "4 "4 Supports up to 4094 PVPs per trunk, with a current maximum of 7000 
Series PVPs per switch. 
8540 MSR Vv Vv Vv Supports up to 256 PVPs per trunk, or 24k per switch. 
BPX 8600 Vv "4 Vv Supports up to 4094 PVPs per trunk, with a current maximum of 
Series 12,000 PVPs per switch. 
M GX 8850 "4 V "4 Supports up to 4096 PVPs per trunk, with a current maximum of 
IP+ATM 12,000 PVPs per switch. 
switch 
Proposed "4 "4 "4 Supports up to 4096 PVPs per trunk, with a maximum in excess of 
M GX 8800with 12,000 PVPs per switch. 
PXM-45 
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ATM-LSRs 
In order to support VP Tunnels, ATM -LSRs must meet the same requirements as ATM EdgeLSRs: 


* They must beable to support distinct logical M PLS links in different sub-interfaces of a physical ATM link. If this requirement is 
not met, then the ATM -LSR would require a separate physical ATM interface for each VP Tunnel. This would create severe 
scalability issues. For example, the ATM -LSR in Figure 16(b) would require five ATM links, whereas the ATM -LSRs in 
Figure 16(d) would require three. If a particular VP tunnel uses a VPI x at the ATM -LSR, then theLDP signalling VC for the VP 
Tunnel must be within x. It may have VPl=x, VC1=32, instead of the normal default VPI=0, VCI=32. 

* They must support shaping of packet traffic entering the VP tunnel to the traffic parameters (e.g. a CBR peak rate) of the VP 
tunnel. It is highly preferable that this be done using Early Packet Discard (EPD), or at least Partial Packet Discard (PPD). 


Table 15 Cisco ATM -LSRs 
VP Tunnel Traffic Smart 
oe Types Supported Discard for 
Minimum Software & | (With Shaping) VPs 
Hardware for VP 
Device Tunnels CBR? | VBR | ABR | EPD | PPD | Capabilities & Limitations 
LS1010 Per-Flow Queueing Feature v x x x x A maximum of 64k LVCs are supported per switch, irrespective of the 
Card Required (FC-PFQ)). VP Tunnels it supports. 
10S 11.3(5)WA4(8b) or later. 64 VP Tunnels are supported. 
6400 Any supported hardware, Vv 4 x x 4 A maximum of 64k LVCs are supported per switch, irrespective of the 
10S 112.0(6)Db VP Tunnels it supports. 
128 VP Tunnels are supported. 
8540 MSR Any supported software Vv x x x x A maximum of 256k LVCs are supported per switch, irrespective of 
release. the VP Tunnels it supports. 
128VP Tunnels are supported. 
BPX 8600 BXM cards with firmware Vv x x v A maximum of 32k LVCs are supported per BXM card, irrespective of 
Series MEB, Switch Software the number of ports or VP Tunnels it supports. 
9.2.50 Each BXM card supports at least 20 VP Tunnel endpoint s (12-port 
T3) and may support up to 31. 
Ifa BXM card is terminating MPLS VP tunnels ona particular BXM 
trunk, it may support no other type of connection on that BXM trunk 
except virtual trunks. 
MGX 8850 VP tunnels are supported Vv x x Vv A maximum of 16k LVCs are supported per M GX 8850 in this 
IP+ATM on PXM-1B cards, when an configuration, irrespective of the number of ports or VP Tunnels it 
switch LSC is used?, supports. 
There is also a dependency Each PXM card supports at least 28 VP Tunnel endpoints and may 
of an LSC interface to the support up to 31. 
PXM-1cards, which may If an M GX8850 is terminating MPLS VP tunnels ona particular PXM 
not occur until H2 CY 2000, trunk, it may support no other type of connection on that PXM trunk. 
as LSC support for the 
PXM-45 is a higher priority. 
Proposed AXSM cards, software and Vv 4 x Vv A maximum of 128k LVCs are supported per AXSM card, irrespective 
MGX 8800with |) firmware to be determined. of the number of ports or VP Tunnels it supports. 
PXM-45 Target support timeframe is Each AXSM card supports at least 49 VP Tunnel endpoint s (16-port 
Q2 CY 2000 E3/T3 or OC3) and may support up to 64. 
If an AXSM card is terminating MPLS VP tunnels on a particular 
trunk, it may support no other type of connection on that trunk 
except virtual trunks. 
a. This means that the shaper will shape to a peak rate. The resulting shaped cell stream is guaranteed to fit within a a CBR PVP, but it is actually a VBR stream with indeterminate 


Sustained Cell Rate (SCR). 


b. If an LSC is not used, then VP tunnels can still terminate on the RPM cards. If an LSC is not used, then the M GX 8850 is acting as separate edge LSRs (Table 13) in combination 
with a traditional ATM switch (Table 14). 


VP Tunnels and the SCI 
The Virtual switch Interface (SCI) was described in “2.3 |P+ATM” on page 14. Recall that part of the configuration of SCI involves 


creating partitions of the bandwidth and VPI/VCI space on trunks. This is shown in Figure 18(a). M PLS VP Tunnels area type of 
Virtual Trunk. Virtual trunks are PVP or SPVP connections which carry VCs through switches that don’t support that type of VC. 
Aswell asM PLS, virtual trunks can tunnel PNNI SVCs through switches that do not support them. In addition they can be used to 
connect a customer's virtual circuits of any type, through a carrier’s switches. 
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AutoRoute 


b. Virtual trunks 


AutoRoute 


c. Virtual trunks between adjacent nodes 


Figure 18 SCI partitions and virtual trunks 


Inthe BPX 8650 and M GX 8850 implementation of virtual trunks, each virtual trunk will beable to have several SCI partitions 
as if it were a physical trunk— see Figure 18(b). This may be supported from BPX software release 9.3 onwards. In practice, the 
number of virtual trunks supported per card in the BPX 8650 and M GX 8850 is limited by the number of Virtual Interfaces 
supported by thecard. A Virtual Interfaceis a physical device which shapes traffic and provides per-VC queueing. Therearea limited 
number of Virtual Interfaces per card. Each BXM, PXM or UXM card has 31 Virtual Interfaces, and an AX SM has 64. They are 
used as follows: 
¢ Each physical interface which does not support virtual trunks, uses one Virtual Interface. For example, the switch ports in 

Figure 18(a) use only one Virtual Interface each, despite having three partitions each. 
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¢ Each endpoint of a virtual trunk uses a Virtual Interface. For example, the interfaces shown in Figure 18(c) use three Virtual 
Interfaces each. 
N ote that there are generally several SCI partitions on each Virtual Interface. Virtual Interfaces are generally not used to shape 
individual VSI partitions. This is because a card can support very many more partitions than Virtual Interfaces. For example, a 
16-port OC3 card on an M GX 8800 platform will eventually support 16 partitions on each of its 16 interfaces, for a total of 1024 
partitions. H owever it supports only 64 Virtual Interfaces. N ote that from the limits described above, this card will support 49 
virtual trunk endpoints if all endpoints are on one physical interface, and up to 64 endpoints if they are distributed over several 
interfaces. 


Customer 
access _ 
lines VP Tunnels 
to another ATM MPLS 
LSR Link(s) 
Edge LSRs Non-M PLS ATM-LSR 
Switch 
a. Edge LSR/switch without MPLS switching b. Edge LSR/switch with ATM -LSR 
Capability (supported on Catalyst 5000 or Capability (soon on 6400 and 
early-release M GX 8850, but not 6400)) M GX 8850, but not Catalyst 5500) 


VP Tunnels 
to other LSRs 


c. Terminating VP Tunnels on the 
ATM-LSR (soon on 6400 and 
M GX 8850, but not Catalyst 5500) 


Figure 19 Devices combining Edge Label Switch Routers and an ATM switch 


Edge LSRs Integrated into Edge Switches 
EdgeLSR function may be integrated into an ATM access switch, as shown in Figure 19. The Cisco M GX 8850, Catalyst 5500, 


and Cisco 6400 are examples of such devices. An Edge LSR/switch may include one or more logically separate edge LSRs, distinct 
from the switch component. In theCisco M GX 8850, the edgeLSRs are R oute Processor M odule(RPM ) cards, in theC atalyst 5500 
they areR oute Switch M odule (RSM ) cards, and in the 6400 they areN odeR oute Processor (N RP) cards. T hereare several possible 
combinations of M PLS function and VP Tunnels in such a device. 
* EdgeLSRs Only: 
If the switch component cannot perform M PLS switching, then VP Tunnd’s (or M PLS-over-PV Cs) must be used to connect the 
EdgeLSRs to other EdgeLSRs, as shown in Figure 19(a). VP Tunnels will also be used to inter-connect the Edge LSRsin the same 
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device. The three Edge LSR components shown are equivalent to three of the Edge LSRs in Figure 16(a), except that they are 

housed in a single chassis. N ote that each Edge LSR component would require a separate VP Tunnel to connect to another ATM 

M PLS device. TheV Ps shown in Figure 19(a) are sufficient to connect to only one other ATM M PLSdevice M orewould be used 

as necessary. The 6400 can use this combination of functions only if M PLS-over-PV Csis used, as theN RPs cannot directly support 

VP tunnels. 

Full LSR plus Edge LSR: 

If the switch component has M PLS control, then the Edge LSR/switch can act as a full ATM -LSR as well as EdgeL SR (s). This is 

shown in Figure 19(b). In the M GX 8850, ATM -LSR capability will be enabled by using oneof the RPM s as a Label Switch 

Controller (LSC) for the switch; in the 6400, the M PLS control function resides on the N ode Switch Processor. An RPM in the 

M GX 8850 acting asan LSC may concurrently act asan Edge LSR. TheATM -LSR component interconnects the EdgeLSRs using 

full M PLS, so VP Tunnels are no longer required within the device. Figure 19(b) shows the case where the Edge LSR /switch 

connects to the rest of the network viaATM MPLS links. An LSR/EdgeLSR performs the functions of several of the devices in 

Figure 16(d) or the left side of Figure 16(c). N ote that the Catalyst 5500 does not support this mode, as it cannot act as a full 

ATM-LSR. 

* Full LSR/EdgeLSR connected with VP Tunnels: 
If necessary, the full LSR/EdgeLSR device can be connected by VP Tunnels to other ATM M PLS devices. This is shown in 
Figure 19(c). Even though VP Tunnels arestill used, having internal M PLS switching simplifies the VP Tunnel mesh. For example, 
the three external VP Tunnels shown in Figure 19(a) are sufficient to connect to only one other ATM LSR or EdgeLSR, but the 
VP Tunnels shown in Figure 19(c) are sufficient to connect to three other devices. This applies only to the 6400 and M GX 
8850— the Catalyst 5500 does not support this. 

The requirements on the components within the LSR/Edge LSR are similar to the requirements for stand-alone devices discussed 

earlier. In particular: 

¢ If the Edge LSR components (RPM s or RSM s) terminate VP Tunnels as in Figure 19(a), then they must meet the requirements 
discussed in “Edge LSRs” on page 44. 

* If the switch component does not have M PLS capability, i.e. it acts as shown in Figure 19(a), then it must meet the requirements 
discussed in “Traditional ATM Switches” on page 47. 

¢ If the switch component has M PLS capability but supports VP Tunnels, i.e. it acts as shown in Figure 19(c), then it must meet the 
requirements discussed in “ATM -LSRs” on page 48. 


VP Tunnel Restrictions and Work-Arounds on the BPX 8650 
TheBPX 8650 has certain restrictions on VP Tunnel use, which may beworked-around. These restrictions do not apply to theM GX 


8850 or any other Cisco ATM switch. The restrictions apply to the use of an ATM interfacein aBPX 8650. An ATM interface can 

be configured in several distinct modes: 

* In either “port” or “trunk” mode, the interface can support any combination of AutoR oute-controlled PVCs and PVPs; SVCs, 
SPV Cs and SPV Ps controlled by the Service Expansion Shelf?; and M PLSLVCs. Theinterface cannot shape VP Tunnels, but it can 
carry the mid-points of VP Tunnels (because the middle of a VP Tunnel is just a PVP connection). 

¢ In “AutoR oute Virtual Trunk” mode, the interface can terminate and shape Virtual Trunks which carry PVC connections only. 
T heinterface or the virtual trunks may not carry M PLS or PNNI connections. T heinterface may not carry PVC connections unless 
they are within the Virtual Trunks. T he interface may not carry PV Ps unless the PV Ps are the Virtual Trunks which terminate on 
this interface. 

¢ In “VSI Virtual Trunk” mode, the interface can terminate and shape Virtual Trunks and VP Tunnels which carry M PLSLVCs and 
PNNISVCs& SPVCs. Theinterface may not carry PVCsatall, and may not carry any connections except those inside the Virtual 
Trunks. The interface may not carry PV Ps unless the PV Ps are the Virtual Trunks which terminate on this interface. 


3. From BPX software release 9.2.50 onwards 
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ATM link carrying several PVPs (VP Tunnels only) 


B. ATM interface carrying PVP 
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Figure 20 VP tunnels alongside ordinary PVC & PVP in the BPX 8650 


These restrictions could prevent the BPX 8650 from carrying Virtual Trunks and VP Tunnels on the same links as ordinary 
AutoR oute, PNNI and M PLS (S)PVCs, (S)PVPs, SVCs and LVCs. H owever there is a work-around, as shown in Figure 20. 

In the work-around, interface A is configured as a VSI Virtual Trunk interface, and it terminates several VP Tunnels. This isthe 
normal way of terminating VP Tunnelson a BPX 8650. However the link connected to A is looped back to another interface B. B 
is configured as a normal Port, and it carries several PVP connections. The PVP connections are the VP Tunnels. B has no special 
configuration for the VP Tunnels—it treats them as ordinary PVPs. The VP Tunnels are switched, as ordinary PVPs, through to 
interface C. Interface C is an ordinary port or trunk, and it can carry any combination of AutoR oute, PN NI and M PLS connections, 
both inside and outside the VP Tunnels. 

M any variations of this work-around are possible. For example, the VP Tunnels can be distributed onto several different 
interfaces, and not just a single interface C. Interface A might be on a different switch to interfaces B and C. 


Use of Inverse Multiplexing over ATM 
Inverse M ultiplexing over ATM (IMA) trunks may be used in the traditional ATM parts of the network. For example, any of the 


links in the ‘clouds in Figure 13 or Figure 16 may beIM A links. This allows!M A to be used in an M PLS network, even if the ATM 
LSRsdo not directly support IMA. 


4.5 Examples of Hybrid ATM Network Equipment 


MGX 8850 Edge LSRs connected over a big cloud 

It is possibleto run an M GX 8850 M PLS network without using AT M -LSRsatall. Therearetwo ways to do this: M PLS-over-PVCs 

and VP Tunnels: 

* M PLS-over-PV Cs: Refer to Figure 13(a). For theedgeLSR, RPMsinM GX 8850 IP+ATM switches will be used. T heseare simply 
connected by PVCs, and ATM -LSRs (and LSCs) are not used in the network. 
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Figure 21 Deployment of a network with M GX 8850s and core ATM -LSRs, but without LSCs on the M GX 8850 


¢ VP Tunnels: Refer to Figure 16(a) and assume that the edge LSRS areRPMSin MGX 8850 IP+ATM switches. According to 
Table 13, this will be supported with up to 256 VP Tunnels per RPM. The ATM cloud must support CBR PVPsor SPV Ps. 


M GX 8850 Deployments Without LSCs in the M GX 8850s 
A more scalable deployment mechanism for MGX 8850s without LSCsis shown in Figure 21. H ere, ATM -LSRs, e.g. BPX 8650s, 


are chosen for the core of the network. VP tunnels are used at the edge of the network to connect RPM sto each other and to the 
BPX 8650s. Because the VP tunnels are kept short, and kept at the edge of the network, the network is quite scalable. The BPX 
8650 is a quite suitable AT M -LSR for this application, because of the excellent VP Tunnel support as shown in Table 15. 


4.6 VP Tunnels and LVC Usage 
Dimensioning of the number of M PLS Label VCs required per link is described in “3.4 Dimensioning M PLS Label VC space” on 


page 30. This dimensioning requires some modification to deal with VP Tunnels. When VP Tunnels are terminate on an interface, 
theLV Csonall VP Tunnels must be taken into account. For example, if 4 VP Tunnels terminate on a logical interface which supports 
4000 LVCs, then an average of only 1000 LVCs will be available per VP Tunnel. This means that Equations (1) to (7) need to be 
modified to deal with VP Tunnels. 
Equation (1), which applies to interfaces on and routers, becomes 
l<cdt, (8) 
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where | is the number of LVCs used per link, d is the number of destination-prefixes known in an area, and c is the number of 
classes-of-service used, and t isthenumber of VP Tunnels terminating on the interface in question. For example, if 500 destinations 
are known in an area, 4 classes of service are supported in an area, and 4 VP tunnels terminate on a physical interface, less than 
8000 LVCs will be required on that physical interface. 

Similarly, equations (2) and (3) also apply to interfaces on routers under different circumstances. They also get a multiplier of 
t, and become 


I<t(c(d—d,) +cnd,). (9) 
and 
l<2tcn (10) 
Equation (4) which applies to switches with VC merge, is the same as equation (1), and is modified in the same way: 
|< cdt, (11) 
Equation (5), which also applies to switches with VC merge, stays the same: 
m<cd(k-—1), (5) 


where m is the number of LVCs which must be merged together in the switch, and k the number of links into the switch. However 
k must include VP Tunnels as well as physical links. 
Equations (6) and (7). which apply to switches without VC merge, also get multipliers according to thenumber of virtual trunks 
on the interfaces, becoming 
l<cd(n—-1)t, (12) 
and 
l<c(n2/2)t. (13) 
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